Max level shown:
12026-03-24T21:00:05.516ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
22026-03-24T21:00:05.516ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
32026-03-24T21:00:05.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42026-03-24T21:00:05.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
52026-03-24T21:00:05.517ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62026-03-24T21:00:05.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
72026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
82026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
92026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
102026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
112026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
122026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
132026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
142026-03-24T21:00:05.520ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2026-03-24 20:50:05.520739710 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
152026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 7 reason = Timeout
162026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
172026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
182026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
192026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
202026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
212026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
222026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
232026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242026-03-24T21:00:05.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
252026-03-24T21:00:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
262026-03-24T21:00:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
272026-03-24T21:00:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
282026-03-24T21:00:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
292026-03-24T21:00:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
302026-03-24T21:00:05.521ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
312026-03-24T21:00:05.521ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
322026-03-24T21:00:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("93b6e989-1f7d-4884-aec9-ecbc076ed6c4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 735240700 } })
332026-03-24T21:00:05.521ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
342026-03-24T21:00:05.521ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = ExternalNetworking step id = Ensure
352026-03-24T21:00:05.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
362026-03-24T21:00:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("27f3a9c5-67c7-41c5-994d-9db513cda557.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2f7ea2ed-fd45-4cd7-9fb1-f48119541118.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 735318122 } })
372026-03-24T21:00:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
382026-03-24T21:00:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.448349ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
392026-03-24T21:00:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
402026-03-24T21:00:05.522ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
412026-03-24T21:00:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
422026-03-24T21:00:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
432026-03-24T21:00:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
442026-03-24T21:00:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.550282ms iteration = 4 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
452026-03-24T21:00:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
462026-03-24T21:00:05.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
472026-03-24T21:00:05.523ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
482026-03-24T21:00:05.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
492026-03-24T21:00:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f58d4fa9-ccaa-4202-9867-0d750bbfc1dc", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
502026-03-24T21:00:05.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
512026-03-24T21:00:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "264e2404-1d58-4209-bc5b-bb285ebf625f", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
522026-03-24T21:00:05.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
532026-03-24T21:00:05.524ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
542026-03-24T21:00:05.524ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 3.600157ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
552026-03-24T21:00:05.525ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2026-03-24 20:50:05.520739710 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
562026-03-24T21:00:05.525ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.266275ms iteration = 4 status = {"expiration":"2026-03-24T20:50:05.520739710Z","pruned":[]}
572026-03-24T21:00:05.525ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
582026-03-24T21:00:05.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "19dcc4a3-d506-4973-9e34-e809b9d90171", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
592026-03-24T21:00:05.525ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
602026-03-24T21:00:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "f321beec-e541-4ffb-a5f9-d878bfb87939", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
612026-03-24T21:00:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
622026-03-24T21:00:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "0f829cde-a315-4f89-926f-91bf61316642", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
632026-03-24T21:00:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
642026-03-24T21:00:05.526ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
652026-03-24T21:00:05.526ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 5.433946ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
662026-03-24T21:00:05.526ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
672026-03-24T21:00:05.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
682026-03-24T21:00:05.527ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 741421276 } })
692026-03-24T21:00:05.527ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
702026-03-24T21:00:05.527ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
712026-03-24T21:00:05.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
722026-03-24T21:00:05.529ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
732026-03-24T21:00:05.530ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 9.25886ms iteration = 7 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
742026-03-24T21:00:05.530ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 744726725 } })
752026-03-24T21:00:05.531ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
762026-03-24T21:00:05.531ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
772026-03-24T21:00:05.531ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
782026-03-24T21:00:05.531ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
792026-03-24T21:00:05.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
802026-03-24T21:00:05.531ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 745533357 } })
812026-03-24T21:00:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
822026-03-24T21:00:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
832026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
842026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Timeout
852026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
862026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
872026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 12 reason = Timeout
882026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
892026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
902026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
912026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 10 reason = Timeout
922026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
932026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
942026-03-24T21:00:05.533ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
952026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
962026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 18 reason = Timeout
972026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
982026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
992026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1002026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 37 reason = Timeout
1012026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
1022026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
1032026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1042026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
1052026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1062026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1072026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1082026-03-24T21:00:05.533ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
1092026-03-24T21:00:05.533ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
1102026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1112026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
1122026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1132026-03-24T21:00:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1142026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1152026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
1162026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1172026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
1182026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
1192026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
1202026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1212026-03-24T21:00:05.534ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "99d6526b-3e66-43de-b141-a0368a800a97", "content-length": "26", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} })
1222026-03-24T21:00:05.534ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
1232026-03-24T21:00:05.534ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
1242026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1252026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1262026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1272026-03-24T21:00:05.534ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 748385824 } })
1282026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
1292026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1302026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1312026-03-24T21:00:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
1322026-03-24T21:00:05.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1332026-03-24T21:00:05.535ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1342026-03-24T21:00:05.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
1352026-03-24T21:00:05.537ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1362026-03-24T21:00:05.537ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1372026-03-24T21:00:05.537ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1382026-03-24T21:00:05.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1392026-03-24T21:00:05.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1402026-03-24T21:00:05.539ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 18.607352ms iteration = 7 status = {}
1412026-03-24T21:00:05.540ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1422026-03-24T21:00:05.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1432026-03-24T21:00:05.541ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1442026-03-24T21:00:05.542ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1452026-03-24T21:00:05.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
1462026-03-24T21:00:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
1472026-03-24T21:00:05.543ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1482026-03-24T21:00:05.543ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1492026-03-24T21:00:05.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1502026-03-24T21:00:05.544ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1512026-03-24T21:00:05.544ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1522026-03-24T21:00:05.544ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1532026-03-24T21:00:05.544ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 24.190632ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
1542026-03-24T21:00:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1552026-03-24T21:00:05.545ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1562026-03-24T21:00:05.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1572026-03-24T21:00:05.546ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1582026-03-24T21:00:05.547ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
1592026-03-24T21:00:05.547ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1602026-03-24T21:00:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
1612026-03-24T21:00:05.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1622026-03-24T21:00:05.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1632026-03-24T21:00:05.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1642026-03-24T21:00:05.549ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1652026-03-24T21:00:05.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1662026-03-24T21:00:05.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
1672026-03-24T21:00:05.550ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1682026-03-24T21:00:05.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1692026-03-24T21:00:05.550ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1702026-03-24T21:00:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1712026-03-24T21:00:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1722026-03-24T21:00:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
1732026-03-24T21:00:05.551ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
1742026-03-24T21:00:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1752026-03-24T21:00:05.551ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1762026-03-24T21:00:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
1772026-03-24T21:00:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
1782026-03-24T21:00:05.552ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
1792026-03-24T21:00:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1802026-03-24T21:00:05.552ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1812026-03-24T21:00:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
1822026-03-24T21:00:05.552ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1832026-03-24T21:00:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
1842026-03-24T21:00:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1852026-03-24T21:00:05.552ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1862026-03-24T21:00:05.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1872026-03-24T21:00:05.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1882026-03-24T21:00:05.553ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1892026-03-24T21:00:05.553ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
1902026-03-24T21:00:05.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
1912026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
1922026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
1932026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
1942026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
1952026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 7 reason = Timeout
1962026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1972026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
1982026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1992026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
2002026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2012026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2022026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
2032026-03-24T21:00:05.554ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
2042026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
2052026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2062026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2072026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 7 reason = Timeout
2082026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2092026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 4 reason = Timeout
2102026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
2112026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
2122026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
2132026-03-24T21:00:05.554ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 768510837 } })
2142026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
2152026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
2162026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
2172026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2182026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
2192026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
2202026-03-24T21:00:05.554ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2212026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
2222026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2232026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
2242026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
2252026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2262026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2272026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2282026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2292026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
2302026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
2312026-03-24T21:00:05.555ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
2322026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 80ec73ed-5e31-4e68-b930-28453af567b6 original_time_started = 2026-03-24 20:59:09.305048 UTC
2332026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 21.972353ms iteration = 18 status = {"Loaded":{"collection_id":"80ec73ed-5e31-4e68-b930-28453af567b6","time_loaded":"2026-03-24T21:00:05.533441152Z","time_started":"2026-03-24T20:59:09.305048Z"}}
2342026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2352026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
2362026-03-24T21:00:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2372026-03-24T21:00:05.555ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 769587626 } })
2382026-03-24T21:00:05.556ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2392026-03-24T21:00:05.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2402026-03-24T21:00:05.556ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2412026-03-24T21:00:05.556ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2422026-03-24T21:00:05.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
2432026-03-24T21:00:05.556ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2442026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2452026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
2462026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2472026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 24.000967ms iteration = 7 status = 0
2482026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.751914ms iteration = 7 status = {"errors":[],"finish_invoked_ok":[]}
2492026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 10 reason = Timeout
2502026-03-24T21:00:05.557ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
2512026-03-24T21:00:05.557ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
2522026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 4 reason = Timeout
2532026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 21.11\u{b5}s iteration = 10 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
2542026-03-24T21:00:05.557ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
2552026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
2562026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 7 reason = Timeout
2572026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
2582026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 4 reason = Timeout
2592026-03-24T21:00:05.557ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
2602026-03-24T21:00:05.558ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2612026-03-24T21:00:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
2622026-03-24T21:00:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
2632026-03-24T21:00:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
2642026-03-24T21:00:05.558ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "381327dc-c1a3-486b-90b2-849ab92b2c13", "content-length": "26", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} })
2652026-03-24T21:00:05.558ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
2662026-03-24T21:00:05.558ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
2672026-03-24T21:00:05.558ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.269525ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
2682026-03-24T21:00:05.559ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2692026-03-24T21:00:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2702026-03-24T21:00:05.559ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2712026-03-24T21:00:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2722026-03-24T21:00:05.560ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 774072147 } })
2732026-03-24T21:00:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2742026-03-24T21:00:05.561ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2752026-03-24T21:00:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2762026-03-24T21:00:05.562ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2772026-03-24T21:00:05.562ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2782026-03-24T21:00:05.562ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2792026-03-24T21:00:05.562ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2802026-03-24T21:00:05.563ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 777040607 } })
2812026-03-24T21:00:05.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2822026-03-24T21:00:05.564ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2832026-03-24T21:00:05.564ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2842026-03-24T21:00:05.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
2852026-03-24T21:00:05.565ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2862026-03-24T21:00:05.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2872026-03-24T21:00:05.565ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2882026-03-24T21:00:05.566ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2892026-03-24T21:00:05.566ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2902026-03-24T21:00:05.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2912026-03-24T21:00:05.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
2922026-03-24T21:00:05.566ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
2932026-03-24T21:00:05.566ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
2942026-03-24T21:00:05.566ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 12.156198ms iteration = 4 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
2952026-03-24T21:00:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
2962026-03-24T21:00:05.567ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
2972026-03-24T21:00:05.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2982026-03-24T21:00:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
2992026-03-24T21:00:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
3002026-03-24T21:00:05.569ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3012026-03-24T21:00:05.569ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
3022026-03-24T21:00:05.569ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 12.24066ms iteration = 4 status = {"by_rx":{},"error":null}
3032026-03-24T21:00:05.570ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3042026-03-24T21:00:05.570ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3052026-03-24T21:00:05.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
3062026-03-24T21:00:05.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
3072026-03-24T21:00:05.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3082026-03-24T21:00:05.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
3092026-03-24T21:00:05.573ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3102026-03-24T21:00:05.573ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3112026-03-24T21:00:05.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
3122026-03-24T21:00:05.576ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 7 reason = Timeout
3132026-03-24T21:00:05.576ZDEBGnexus (BackgroundTasks): activating background_task = trust_quorum_manager iteration = 4 reason = Timeout
3142026-03-24T21:00:05.576ZDEBGnexus (BackgroundTasks): activating background_task = attached_subnet_manager iteration = 4 reason = Timeout
3152026-03-24T21:00:05.576ZINFOnexus (BackgroundTasks): starting attached subnet manager work background_task = attached_subnet_manager file = nexus/src/app/background/tasks/attached_subnets.rs:374
3162026-03-24T21:00:05.576ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 18.121\u{b5}s iteration = 7 status = {"disabled":true,"errors":[],"sps":[]}
3172026-03-24T21:00:05.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 13 reason = Timeout
3182026-03-24T21:00:05.576ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
3192026-03-24T21:00:05.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
3202026-03-24T21:00:05.576ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 4 reason = Timeout
3212026-03-24T21:00:05.576ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
3222026-03-24T21:00:05.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 13 reason = Timeout
3232026-03-24T21:00:05.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
3242026-03-24T21:00:05.576ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
3252026-03-24T21:00:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
3262026-03-24T21:00:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3272026-03-24T21:00:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
3282026-03-24T21:00:05.577ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 991, tv_nsec: 790930102 } })
3292026-03-24T21:00:05.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
3302026-03-24T21:00:05.579ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3312026-03-24T21:00:05.579ZDEBGnexus (BackgroundTasks): roles background_task = attached_subnet_manager roles = RoleSet { roles: {} }
3322026-03-24T21:00:05.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = attached_subnet_manager resource = Database result = Ok(())
3332026-03-24T21:00:05.582ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3342026-03-24T21:00:05.582ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3352026-03-24T21:00:05.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3362026-03-24T21:00:05.588ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3372026-03-24T21:00:05.588ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
3382026-03-24T21:00:05.588ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3392026-03-24T21:00:05.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
3402026-03-24T21:00:05.592ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
3412026-03-24T21:00:05.592ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3422026-03-24T21:00:05.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3432026-03-24T21:00:05.593ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 59.855114ms iteration = 3 status = {"nsuccess":5}
3442026-03-24T21:00:05.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3452026-03-24T21:00:05.599ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
3462026-03-24T21:00:05.599ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3472026-03-24T21:00:05.599ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2026-03-24 20:57:05.047625 UTC dns_group = internal generation = 1
3482026-03-24T21:00:05.599ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 83.44219ms iteration = 4 status = {"generation":1}
3492026-03-24T21:00:05.599ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
3502026-03-24T21:00:05.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
3512026-03-24T21:00:05.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
3522026-03-24T21:00:05.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
3532026-03-24T21:00:05.600ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
3542026-03-24T21:00:05.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
3552026-03-24T21:00:05.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
3562026-03-24T21:00:05.603ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3572026-03-24T21:00:05.604ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
3582026-03-24T21:00:05.604ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3592026-03-24T21:00:05.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3602026-03-24T21:00:05.607ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
3612026-03-24T21:00:05.607ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
3622026-03-24T21:00:05.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
3632026-03-24T21:00:05.607ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Fleet result = Ok(())
3642026-03-24T21:00:05.608ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
3652026-03-24T21:00:05.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
3662026-03-24T21:00:05.611ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
3672026-03-24T21:00:05.611ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 56.789701ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
3682026-03-24T21:00:05.617ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3692026-03-24T21:00:05.617ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3702026-03-24T21:00:05.620ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
3712026-03-24T21:00:05.621ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
3722026-03-24T21:00:05.621ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
3732026-03-24T21:00:05.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
3742026-03-24T21:00:05.622ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3752026-03-24T21:00:05.622ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3762026-03-24T21:00:05.632ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
3772026-03-24T21:00:05.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3782026-03-24T21:00:05.638ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
3792026-03-24T21:00:05.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3802026-03-24T21:00:05.640ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3812026-03-24T21:00:05.643ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
3822026-03-24T21:00:05.643ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
3832026-03-24T21:00:05.643ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 109.651377ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3842026-03-24T21:00:05.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3852026-03-24T21:00:05.645ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3862026-03-24T21:00:05.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3872026-03-24T21:00:05.646ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3882026-03-24T21:00:05.646ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3892026-03-24T21:00:05.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
3902026-03-24T21:00:05.650ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 92.659069ms iteration = 4 status = {"dispatched":[],"errors":[],"glob_version":"242.0.0","globs_reprocessed":{},"no_receivers":[]}
3912026-03-24T21:00:05.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
3922026-03-24T21:00:05.650ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3932026-03-24T21:00:05.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3942026-03-24T21:00:05.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3952026-03-24T21:00:05.651ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f92975fb-e687-4a31-92cb-febc7231a934", "content-length": "372", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} })
3962026-03-24T21:00:05.651ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3972026-03-24T21:00:05.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3982026-03-24T21:00:05.653ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3992026-03-24T21:00:05.653ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
4002026-03-24T21:00:05.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4012026-03-24T21:00:05.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4022026-03-24T21:00:05.655ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4032026-03-24T21:00:05.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
4042026-03-24T21:00:05.658ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
4052026-03-24T21:00:05.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
4062026-03-24T21:00:05.659ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
4072026-03-24T21:00:05.659ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
4082026-03-24T21:00:05.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
4092026-03-24T21:00:05.660ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 83.544843ms iteration = 13 status = "NoSitrep"
4102026-03-24T21:00:05.660ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4112026-03-24T21:00:05.662ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
4122026-03-24T21:00:05.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
4132026-03-24T21:00:05.662ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
4142026-03-24T21:00:05.667ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4152026-03-24T21:00:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4162026-03-24T21:00:05.669ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
4172026-03-24T21:00:05.669ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4182026-03-24T21:00:05.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
4192026-03-24T21:00:05.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4202026-03-24T21:00:05.672ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 117.334784ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
4212026-03-24T21:00:05.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4222026-03-24T21:00:05.673ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
4232026-03-24T21:00:05.673ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
4242026-03-24T21:00:05.673ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 97.346775ms iteration = 13 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
4252026-03-24T21:00:05.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4262026-03-24T21:00:05.674ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
4272026-03-24T21:00:05.674ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
4282026-03-24T21:00:05.675ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
4292026-03-24T21:00:05.675ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
4302026-03-24T21:00:05.675ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
4312026-03-24T21:00:05.675ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
4322026-03-24T21:00:05.675ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
4332026-03-24T21:00:05.675ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4342026-03-24T21:00:05.676ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
4352026-03-24T21:00:05.676ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 142.594685ms iteration = 37 status = {"config_updated":false}
4362026-03-24T21:00:05.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4372026-03-24T21:00:05.676ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
4382026-03-24T21:00:05.676ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 129.666486ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4392026-03-24T21:00:05.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
4402026-03-24T21:00:05.677ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4412026-03-24T21:00:05.677ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
4422026-03-24T21:00:05.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4432026-03-24T21:00:05.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4442026-03-24T21:00:05.678ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4452026-03-24T21:00:05.680ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
4462026-03-24T21:00:05.680ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
4472026-03-24T21:00:05.680ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4482026-03-24T21:00:05.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
4492026-03-24T21:00:05.681ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
4502026-03-24T21:00:05.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
4512026-03-24T21:00:05.682ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4522026-03-24T21:00:05.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4532026-03-24T21:00:05.683ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
4542026-03-24T21:00:05.683ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
4552026-03-24T21:00:05.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
4562026-03-24T21:00:05.685ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
4572026-03-24T21:00:05.685ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 151.863684ms iteration = 19 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
4582026-03-24T21:00:05.687ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
4592026-03-24T21:00:05.690ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
4602026-03-24T21:00:05.690ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
4612026-03-24T21:00:05.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 136.268054ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4622026-03-24T21:00:05.691ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e4ee1556-2d97-47cc-80bf-c69161307b6a", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} })
4632026-03-24T21:00:05.691ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4642026-03-24T21:00:05.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4652026-03-24T21:00:05.692ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
4662026-03-24T21:00:05.695ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4672026-03-24T21:00:05.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4682026-03-24T21:00:05.697ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
4692026-03-24T21:00:05.699ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
4702026-03-24T21:00:05.700ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
4712026-03-24T21:00:05.701ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 167.968339ms iteration = 13 status = {"physical_disks_added":0}
4722026-03-24T21:00:05.701ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4732026-03-24T21:00:05.701ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
4742026-03-24T21:00:05.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4752026-03-24T21:00:05.703ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
4762026-03-24T21:00:05.703ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2026-03-24T20:56:59.957750Z, time_modified: 2026-03-24T20:56:59.957750Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }) }
4772026-03-24T21:00:05.703ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 182.982814ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]}},"default_endpoint":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
4782026-03-24T21:00:05.704ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 147.200299ms iteration = 7 status = {"errors":[],"requests_created_ok":[]}
4792026-03-24T21:00:05.705ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
4802026-03-24T21:00:05.705ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
4812026-03-24T21:00:05.706ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
4822026-03-24T21:00:05.707ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
4832026-03-24T21:00:05.709ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "9888f3c2-18b5-4734-aa76-ed70aa35467d", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} })
4842026-03-24T21:00:05.709ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
4852026-03-24T21:00:05.709ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
4862026-03-24T21:00:05.709ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 133.382667ms iteration = 4 status = {"errors":[],"probes_by_sled":{"b4293d4f-3308-4dfa-b261-a103215913e4":0}}
4872026-03-24T21:00:05.710ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
4882026-03-24T21:00:05.710ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "4d2a8cfe-c9e2-4abf-adc4-58d40d0edfe1", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} })
4892026-03-24T21:00:05.711ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 156.359206ms iteration = 10 status = {}
4902026-03-24T21:00:05.711ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Dependency
4912026-03-24T21:00:05.711ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
4922026-03-24T21:00:05.711ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4932026-03-24T21:00:05.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4942026-03-24T21:00:05.714ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
4952026-03-24T21:00:05.717ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4962026-03-24T21:00:05.720ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
4972026-03-24T21:00:05.720ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
4982026-03-24T21:00:05.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
4992026-03-24T21:00:05.724ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
5002026-03-24T21:00:05.724ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5012026-03-24T21:00:05.727ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
5022026-03-24T21:00:05.727ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5032026-03-24T21:00:05.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5042026-03-24T21:00:05.728ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
5052026-03-24T21:00:05.729ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = SledList step id = Fetch
5062026-03-24T21:00:05.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5072026-03-24T21:00:05.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5082026-03-24T21:00:05.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5092026-03-24T21:00:05.737ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
5102026-03-24T21:00:05.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5112026-03-24T21:00:05.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5122026-03-24T21:00:05.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5132026-03-24T21:00:05.742ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
5142026-03-24T21:00:05.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5152026-03-24T21:00:05.743ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 188.40628ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
5162026-03-24T21:00:05.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5172026-03-24T21:00:05.744ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
5182026-03-24T21:00:05.744ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
5192026-03-24T21:00:05.744ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5202026-03-24T21:00:05.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5212026-03-24T21:00:05.746ZINFOnexus (BackgroundTasks): Loaded 0 active trust quorum configurations from database background_task = trust_quorum_manager bgtask = TrustQuorumManager file = nexus/src/app/background/tasks/trust_quorum.rs:83
5222026-03-24T21:00:05.746ZDEBGnexus (BackgroundTasks): activation complete background_task = trust_quorum_manager elapsed = 170.343014ms iteration = 4 status = {"PerRackStatus":{"errors":[],"statuses":[]}}
5232026-03-24T21:00:05.750ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5242026-03-24T21:00:05.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
5252026-03-24T21:00:05.754ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
5262026-03-24T21:00:05.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
5272026-03-24T21:00:05.757ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
5282026-03-24T21:00:05.757ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
5292026-03-24T21:00:05.757ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 202.440578ms iteration = 7 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
5302026-03-24T21:00:05.757ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
5312026-03-24T21:00:05.757ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
5322026-03-24T21:00:05.757ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2026-03-24 20:57:07.125641 UTC dns_group = external generation = 2
5332026-03-24T21:00:05.757ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 237.010691ms iteration = 4 status = {"generation":2}
5342026-03-24T21:00:05.758ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 203.729334ms iteration = 7 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
5352026-03-24T21:00:05.759ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
5362026-03-24T21:00:05.759ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 226.024165ms iteration = 10 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
5372026-03-24T21:00:05.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5382026-03-24T21:00:05.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5392026-03-24T21:00:05.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5402026-03-24T21:00:05.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5412026-03-24T21:00:05.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5422026-03-24T21:00:05.782ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
5432026-03-24T21:00:05.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5442026-03-24T21:00:05.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5452026-03-24T21:00:05.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5462026-03-24T21:00:05.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5472026-03-24T21:00:05.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5482026-03-24T21:00:05.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
5492026-03-24T21:00:05.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5502026-03-24T21:00:05.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5512026-03-24T21:00:05.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5522026-03-24T21:00:05.804ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
5532026-03-24T21:00:05.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5542026-03-24T21:00:05.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5552026-03-24T21:00:05.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5562026-03-24T21:00:05.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5572026-03-24T21:00:05.809ZDEBGnexus (BackgroundTasks): listed attached subnets background_task = attached_subnet_manager n_subnets = 0
5582026-03-24T21:00:05.809ZDEBGnexus (BackgroundTasks): activation complete background_task = attached_subnet_manager elapsed = 233.159617ms iteration = 4 status = {"db_error":null,"dendrite":{"switch0":{"errors":[],"n_subnets_added":0,"n_subnets_removed":0,"n_total_subnets":0}},"sled":{}}
5592026-03-24T21:00:05.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5602026-03-24T21:00:05.814ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
5612026-03-24T21:00:05.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5622026-03-24T21:00:05.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5632026-03-24T21:00:05.815ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
5642026-03-24T21:00:05.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
5652026-03-24T21:00:05.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5662026-03-24T21:00:05.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5672026-03-24T21:00:05.818ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:118 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
5682026-03-24T21:00:05.818ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 263.570287ms iteration = 4 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
5692026-03-24T21:00:05.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5702026-03-24T21:00:05.821ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
5712026-03-24T21:00:05.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5722026-03-24T21:00:05.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5732026-03-24T21:00:05.822ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5742026-03-24T21:00:05.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
5752026-03-24T21:00:05.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5762026-03-24T21:00:05.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5772026-03-24T21:00:05.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5782026-03-24T21:00:05.832ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
5792026-03-24T21:00:05.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5802026-03-24T21:00:05.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5812026-03-24T21:00:05.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5822026-03-24T21:00:05.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5832026-03-24T21:00:05.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5842026-03-24T21:00:05.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5852026-03-24T21:00:05.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5862026-03-24T21:00:05.856ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
5872026-03-24T21:00:05.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5882026-03-24T21:00:05.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5892026-03-24T21:00:05.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5902026-03-24T21:00:05.861ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
5912026-03-24T21:00:05.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5922026-03-24T21:00:05.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5932026-03-24T21:00:05.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
5942026-03-24T21:00:05.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
5952026-03-24T21:00:05.866ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = DeployNexusRecords step id = Ensure
5962026-03-24T21:00:05.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
5972026-03-24T21:00:05.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
5982026-03-24T21:00:05.866ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
5992026-03-24T21:00:05.866ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6002026-03-24T21:00:05.867ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cfe022c-0c27-42f0-97e2-50a1672ee4b0", "content-length": "372", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} })
6012026-03-24T21:00:05.867ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6022026-03-24T21:00:05.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6032026-03-24T21:00:05.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6042026-03-24T21:00:05.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6052026-03-24T21:00:05.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6062026-03-24T21:00:05.869ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
6072026-03-24T21:00:05.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6082026-03-24T21:00:05.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6092026-03-24T21:00:05.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6102026-03-24T21:00:05.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6112026-03-24T21:00:05.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6122026-03-24T21:00:05.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6132026-03-24T21:00:05.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
6142026-03-24T21:00:05.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6152026-03-24T21:00:05.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6162026-03-24T21:00:05.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6172026-03-24T21:00:05.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
6182026-03-24T21:00:05.880ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
6192026-03-24T21:00:05.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6202026-03-24T21:00:05.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6212026-03-24T21:00:05.883ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6222026-03-24T21:00:05.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
6232026-03-24T21:00:05.886ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6242026-03-24T21:00:05.886ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
6252026-03-24T21:00:05.886ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
6262026-03-24T21:00:05.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6272026-03-24T21:00:05.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6282026-03-24T21:00:05.887ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e3f49c77-a66f-470f-8807-d5208375b381", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} })
6292026-03-24T21:00:05.887ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6302026-03-24T21:00:05.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6312026-03-24T21:00:05.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6322026-03-24T21:00:05.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6332026-03-24T21:00:05.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6342026-03-24T21:00:05.889ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6352026-03-24T21:00:05.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
6362026-03-24T21:00:05.890ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6372026-03-24T21:00:05.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6382026-03-24T21:00:05.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6392026-03-24T21:00:05.890ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6402026-03-24T21:00:05.892ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6412026-03-24T21:00:05.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6422026-03-24T21:00:05.893ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
6432026-03-24T21:00:05.893ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
6442026-03-24T21:00:05.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
6452026-03-24T21:00:05.894ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
6462026-03-24T21:00:05.896ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
6472026-03-24T21:00:05.896ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6482026-03-24T21:00:05.897ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
6492026-03-24T21:00:05.898ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a1dbcff1-2525-4c9c-8297-2455e01f1a65", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} })
6502026-03-24T21:00:05.898ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 187.020323ms iteration = 11 status = {}
6512026-03-24T21:00:05.898ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c40a6e88-3d92-4c92-a3ed-1ed61a83b8be", "content-length": "691", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} })
6522026-03-24T21:00:05.898ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
6532026-03-24T21:00:05.898ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
6542026-03-24T21:00:05.898ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
6552026-03-24T21:00:05.898ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 364.986011ms iteration = 7 status = {}
6562026-03-24T21:00:05.932ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = SledAgent step id = Ensure
6572026-03-24T21:00:05.933ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
6582026-03-24T21:00:05.937ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "4417e40d-f818-407f-a49b-73eec45e307a", "date": "Tue, 24 Mar 2026 21:00:05 GMT"} }) sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
6592026-03-24T21:00:05.937ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = FirewallRules step id = Ensure
6602026-03-24T21:00:05.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6612026-03-24T21:00:05.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6622026-03-24T21:00:05.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6632026-03-24T21:00:05.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6642026-03-24T21:00:05.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6652026-03-24T21:00:05.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6662026-03-24T21:00:05.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6672026-03-24T21:00:05.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6682026-03-24T21:00:05.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6692026-03-24T21:00:05.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6702026-03-24T21:00:05.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6712026-03-24T21:00:05.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6722026-03-24T21:00:05.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6732026-03-24T21:00:05.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6742026-03-24T21:00:05.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6752026-03-24T21:00:05.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6762026-03-24T21:00:05.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6772026-03-24T21:00:05.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6782026-03-24T21:00:05.953ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
6792026-03-24T21:00:05.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6802026-03-24T21:00:05.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6812026-03-24T21:00:05.957ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
6822026-03-24T21:00:05.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6832026-03-24T21:00:05.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6842026-03-24T21:00:05.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6852026-03-24T21:00:05.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6862026-03-24T21:00:05.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6872026-03-24T21:00:05.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6882026-03-24T21:00:05.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6892026-03-24T21:00:05.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
6902026-03-24T21:00:05.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
6912026-03-24T21:00:05.963ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
6922026-03-24T21:00:05.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6932026-03-24T21:00:05.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6942026-03-24T21:00:05.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
6952026-03-24T21:00:05.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
6962026-03-24T21:00:05.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6972026-03-24T21:00:05.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6982026-03-24T21:00:05.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
6992026-03-24T21:00:05.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7002026-03-24T21:00:05.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7012026-03-24T21:00:05.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7022026-03-24T21:00:05.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7032026-03-24T21:00:05.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7042026-03-24T21:00:05.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7052026-03-24T21:00:05.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7062026-03-24T21:00:05.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7072026-03-24T21:00:05.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7082026-03-24T21:00:05.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7092026-03-24T21:00:05.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7102026-03-24T21:00:05.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7112026-03-24T21:00:05.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7122026-03-24T21:00:05.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7132026-03-24T21:00:05.979ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
7142026-03-24T21:00:05.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7152026-03-24T21:00:05.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7162026-03-24T21:00:05.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7172026-03-24T21:00:05.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7182026-03-24T21:00:05.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7192026-03-24T21:00:05.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7202026-03-24T21:00:05.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7212026-03-24T21:00:05.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7222026-03-24T21:00:05.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7232026-03-24T21:00:05.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7242026-03-24T21:00:05.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7252026-03-24T21:00:05.987ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
7262026-03-24T21:00:05.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7272026-03-24T21:00:05.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7282026-03-24T21:00:05.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7292026-03-24T21:00:05.992ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
7302026-03-24T21:00:05.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7312026-03-24T21:00:05.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7322026-03-24T21:00:05.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7332026-03-24T21:00:05.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7342026-03-24T21:00:05.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7352026-03-24T21:00:05.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7362026-03-24T21:00:05.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7372026-03-24T21:00:06.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7382026-03-24T21:00:06.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7392026-03-24T21:00:06.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7402026-03-24T21:00:06.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7412026-03-24T21:00:06.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7422026-03-24T21:00:06.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7432026-03-24T21:00:06.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7442026-03-24T21:00:06.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7452026-03-24T21:00:06.012ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
7462026-03-24T21:00:06.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7472026-03-24T21:00:06.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7482026-03-24T21:00:06.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7492026-03-24T21:00:06.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7502026-03-24T21:00:06.018ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7512026-03-24T21:00:06.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7522026-03-24T21:00:06.020ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7532026-03-24T21:00:06.027ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
7542026-03-24T21:00:06.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7552026-03-24T21:00:06.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7562026-03-24T21:00:06.030ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "19396f07-5d3e-4c93-8788-85809a9073b4", "date": "Tue, 24 Mar 2026 21:00:06 GMT"} })
7572026-03-24T21:00:06.030ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
7582026-03-24T21:00:06.030ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7592026-03-24T21:00:06.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7602026-03-24T21:00:06.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7612026-03-24T21:00:06.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7622026-03-24T21:00:06.034ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7632026-03-24T21:00:06.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7642026-03-24T21:00:06.034ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7652026-03-24T21:00:06.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7662026-03-24T21:00:06.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7672026-03-24T21:00:06.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7682026-03-24T21:00:06.037ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7692026-03-24T21:00:06.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7702026-03-24T21:00:06.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7712026-03-24T21:00:06.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7722026-03-24T21:00:06.040ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7732026-03-24T21:00:06.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7742026-03-24T21:00:06.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7752026-03-24T21:00:06.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7762026-03-24T21:00:06.042ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 508.703847ms iteration = 7 status = {}
7772026-03-24T21:00:06.042ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
7782026-03-24T21:00:06.042ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
7792026-03-24T21:00:06.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7802026-03-24T21:00:06.042ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = ff7c18df-a269-4136-9c4a-62dba68d4efc parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
7812026-03-24T21:00:06.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7822026-03-24T21:00:06.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7832026-03-24T21:00:06.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7842026-03-24T21:00:06.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7852026-03-24T21:00:06.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7862026-03-24T21:00:06.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7872026-03-24T21:00:06.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7882026-03-24T21:00:06.050ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
7892026-03-24T21:00:06.050ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7902026-03-24T21:00:06.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
7912026-03-24T21:00:06.053ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
7922026-03-24T21:00:06.053ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
7932026-03-24T21:00:06.053ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 520.460445ms iteration = 12 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
7942026-03-24T21:00:06.055ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
7952026-03-24T21:00:06.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7962026-03-24T21:00:06.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7972026-03-24T21:00:06.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
7982026-03-24T21:00:06.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
7992026-03-24T21:00:06.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8002026-03-24T21:00:06.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8012026-03-24T21:00:06.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8022026-03-24T21:00:06.064ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
8032026-03-24T21:00:06.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8042026-03-24T21:00:06.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8052026-03-24T21:00:06.071ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
8062026-03-24T21:00:06.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8072026-03-24T21:00:06.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8082026-03-24T21:00:06.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8092026-03-24T21:00:06.074ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
8102026-03-24T21:00:06.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8112026-03-24T21:00:06.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8122026-03-24T21:00:06.075ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
8132026-03-24T21:00:06.076ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
8142026-03-24T21:00:06.076ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
8152026-03-24T21:00:06.148ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
8162026-03-24T21:00:06.148ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
8172026-03-24T21:00:06.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8182026-03-24T21:00:06.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8192026-03-24T21:00:06.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8202026-03-24T21:00:06.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8212026-03-24T21:00:06.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8222026-03-24T21:00:06.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
8232026-03-24T21:00:06.157ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
8242026-03-24T21:00:06.157ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "705da642-d085-4fb7-a891-73eb0e4fd44b", "date": "Tue, 24 Mar 2026 21:00:06 GMT"} })
8252026-03-24T21:00:06.157ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
8262026-03-24T21:00:06.157ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = Dns step id = Ensure
8272026-03-24T21:00:06.158ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
8282026-03-24T21:00:06.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8292026-03-24T21:00:06.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8302026-03-24T21:00:06.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8312026-03-24T21:00:06.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8322026-03-24T21:00:06.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8332026-03-24T21:00:06.164ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
8342026-03-24T21:00:06.165ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
8352026-03-24T21:00:06.165ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
8362026-03-24T21:00:06.165ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
8372026-03-24T21:00:06.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8382026-03-24T21:00:06.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8392026-03-24T21:00:06.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8402026-03-24T21:00:06.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
8412026-03-24T21:00:06.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8422026-03-24T21:00:06.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8432026-03-24T21:00:06.172ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
8442026-03-24T21:00:06.172ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
8452026-03-24T21:00:06.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8462026-03-24T21:00:06.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8472026-03-24T21:00:06.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8482026-03-24T21:00:06.177ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
8492026-03-24T21:00:06.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8502026-03-24T21:00:06.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8512026-03-24T21:00:06.179ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
8522026-03-24T21:00:06.179ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
8532026-03-24T21:00:06.179ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
8542026-03-24T21:00:06.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8552026-03-24T21:00:06.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8562026-03-24T21:00:06.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8572026-03-24T21:00:06.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8582026-03-24T21:00:06.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8592026-03-24T21:00:06.184ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
8602026-03-24T21:00:06.185ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
8612026-03-24T21:00:06.185ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
8622026-03-24T21:00:06.185ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
8632026-03-24T21:00:06.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8642026-03-24T21:00:06.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8652026-03-24T21:00:06.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8662026-03-24T21:00:06.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
8672026-03-24T21:00:06.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8682026-03-24T21:00:06.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8692026-03-24T21:00:06.192ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
8702026-03-24T21:00:06.192ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
8712026-03-24T21:00:06.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8722026-03-24T21:00:06.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8732026-03-24T21:00:06.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8742026-03-24T21:00:06.198ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
8752026-03-24T21:00:06.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8762026-03-24T21:00:06.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8772026-03-24T21:00:06.201ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
8782026-03-24T21:00:06.201ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
8792026-03-24T21:00:06.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8802026-03-24T21:00:06.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8812026-03-24T21:00:06.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8822026-03-24T21:00:06.205ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
8832026-03-24T21:00:06.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8842026-03-24T21:00:06.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8852026-03-24T21:00:06.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8862026-03-24T21:00:06.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8872026-03-24T21:00:06.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
8882026-03-24T21:00:06.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
8892026-03-24T21:00:06.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8902026-03-24T21:00:06.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
8912026-03-24T21:00:06.214ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
8922026-03-24T21:00:06.214ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
8932026-03-24T21:00:06.214ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = OmicronZones step id = Cleanup
8942026-03-24T21:00:06.214ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = OmicronZones step id = Cleanup
8952026-03-24T21:00:06.214ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = PhysicalDisks step id = Cleanup
8962026-03-24T21:00:06.214ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = Clickhouse step id = Ensure
8972026-03-24T21:00:06.214ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = Clickhouse step id = Ensure
8982026-03-24T21:00:06.215ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
8992026-03-24T21:00:06.220ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "c57bc14a-ca60-43ba-a0bf-226fc798ecc5", "date": "Tue, 24 Mar 2026 21:00:06 GMT"} })
9002026-03-24T21:00:06.220ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = SupportBundles step id = Cleanup
9012026-03-24T21:00:06.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9022026-03-24T21:00:06.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9032026-03-24T21:00:06.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9042026-03-24T21:00:06.223ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
9052026-03-24T21:00:06.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9062026-03-24T21:00:06.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9072026-03-24T21:00:06.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9082026-03-24T21:00:06.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9092026-03-24T21:00:06.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9102026-03-24T21:00:06.229ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
9112026-03-24T21:00:06.273ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = OmicronZones step id = Cleanup
9122026-03-24T21:00:06.273ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
9132026-03-24T21:00:06.273ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
9142026-03-24T21:00:06.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9152026-03-24T21:00:06.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9162026-03-24T21:00:06.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9172026-03-24T21:00:06.276ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
9182026-03-24T21:00:06.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9192026-03-24T21:00:06.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9202026-03-24T21:00:06.325ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
9212026-03-24T21:00:06.325ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9e1c09c1-3960-4ecf-a904-19e137699ab3 (blueprint), false)
9222026-03-24T21:00:06.325ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = Cockroach step id = Ensure
9232026-03-24T21:00:06.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
9242026-03-24T21:00:06.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
9252026-03-24T21:00:06.344ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
9262026-03-24T21:00:06.345ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f0737f52-c337-4c43-8398-25fc680d5905 step component = MgsUpdates step id = Ensure
9272026-03-24T21:00:06.345ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 6 reason = Dependency
9282026-03-24T21:00:06.345ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 11 reason = Dependency
9292026-03-24T21:00:06.345ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9302026-03-24T21:00:06.345ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9312026-03-24T21:00:06.345ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 823.841875ms iteration = 5 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3040,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":207581507,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":207581457,"secs":0}},"event_index":1,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":207595707,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":137027134,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":137027094,"secs":0}},"event_index":2,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":344637042,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":66550515,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":66550475,"secs":0}},"event_index":3,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":411202577,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4866501,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4866461,"secs":0}},"event_index":4,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":416086409,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":220377573,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":220377543,"secs":0}},"event_index":5,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":636473052,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":56643467,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":56643427,"secs":0}},"event_index":6,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":693124479,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6200,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":6160,"secs":0}},"event_index":7,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":693148080,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2550,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2510,"secs":0}},"event_index":8,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":693153160,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3680,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3630,"secs":0}},"event_index":9,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":693158590,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2060,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2011,"secs":0}},"event_index":10,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":693167160,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6219237,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":6219197,"secs":0}},"event_index":11,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":699387498,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":52186727,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":52186677,"secs":0}},"event_index":12,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":751579326,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":52129326,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":52129276,"secs":0}},"event_index":13,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":803726752,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":19737563,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":19737523,"secs":0}},"event_index":14,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":823478765,"secs":0}},{"data":{"attempt_elapsed":{"nanos":9970,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":9920,"secs":0}},"event_index":15,"execution_id":"caf55b6d-f3e3-41b4-8a51-535965f8547a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":823508415,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3"}
9322026-03-24T21:00:06.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9332026-03-24T21:00:06.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
9342026-03-24T21:00:06.347ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9352026-03-24T21:00:06.347ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9362026-03-24T21:00:06.349ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
9372026-03-24T21:00:06.349ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9382026-03-24T21:00:06.349ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
9392026-03-24T21:00:06.350ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9402026-03-24T21:00:06.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9412026-03-24T21:00:06.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
9422026-03-24T21:00:06.351ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9432026-03-24T21:00:06.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9442026-03-24T21:00:06.353ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9452026-03-24T21:00:06.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
9462026-03-24T21:00:06.355ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
9472026-03-24T21:00:06.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
9482026-03-24T21:00:06.356ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(dff0f708-331b-443c-8e43-71f66e35b97e, 0), (21de1cec-200f-4895-9257-f5ccf99230ad, 0), (78dc662b-7426-4ed1-8bf8-1612ed24b740, 0), (169e21c7-693b-458e-b4c2-84652769bc8b, 0)] collection_id = dff0f708-331b-443c-8e43-71f66e35b97e
9492026-03-24T21:00:06.356ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9502026-03-24T21:00:06.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
9512026-03-24T21:00:06.357ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
9522026-03-24T21:00:06.357ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.879437ms iteration = 11 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
9532026-03-24T21:00:06.689ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = dff0f708-331b-443c-8e43-71f66e35b97e file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
9542026-03-24T21:00:06.689ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9552026-03-24T21:00:06.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
9562026-03-24T21:00:06.693ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(21de1cec-200f-4895-9257-f5ccf99230ad, 0), (78dc662b-7426-4ed1-8bf8-1612ed24b740, 0), (169e21c7-693b-458e-b4c2-84652769bc8b, 0), (9e2d3a04-ad9f-482d-920e-750ede9ddc94, 0)] collection_id = 21de1cec-200f-4895-9257-f5ccf99230ad
9572026-03-24T21:00:06.693ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9582026-03-24T21:00:06.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
9592026-03-24T21:00:07.044ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 21de1cec-200f-4895-9257-f5ccf99230ad file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
9602026-03-24T21:00:07.044ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9612026-03-24T21:00:07.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
9622026-03-24T21:00:07.049ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(78dc662b-7426-4ed1-8bf8-1612ed24b740, 0), (169e21c7-693b-458e-b4c2-84652769bc8b, 0), (9e2d3a04-ad9f-482d-920e-750ede9ddc94, 0), (80ec73ed-5e31-4e68-b930-28453af567b6, 0)] collection_id = 78dc662b-7426-4ed1-8bf8-1612ed24b740
9632026-03-24T21:00:07.049ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9642026-03-24T21:00:07.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
9652026-03-24T21:00:07.153ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 78dc662b-7426-4ed1-8bf8-1612ed24b740 file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
9662026-03-24T21:00:07.154ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
9672026-03-24T21:00:07.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
9682026-03-24T21:00:07.158ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(169e21c7-693b-458e-b4c2-84652769bc8b, 0), (9e2d3a04-ad9f-482d-920e-750ede9ddc94, 0), (80ec73ed-5e31-4e68-b930-28453af567b6, 0)]
9692026-03-24T21:00:07.158ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 993, tv_nsec: 372592360 } })
9702026-03-24T21:00:07.160ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1345db7a-abc1-4a4c-a49c-e6c56102e521.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1920db37-87f5-4a96-b3ef-1db2eb59e9a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6b02d2eb-4373-4f12-8efc-4c0e3f4fa77b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("709c4704-f768-4640-a198-a23621ed6e11.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 993, tv_nsec: 374427429 } })
9712026-03-24T21:00:07.161ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
9722026-03-24T21:00:07.161ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
9732026-03-24T21:00:07.161ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
9742026-03-24T21:00:07.161ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f1e73cf-78a4-4da6-9edc-a5fe7d7ed768", "content-length": "476", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9752026-03-24T21:00:07.161ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
9762026-03-24T21:00:07.162ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a582bdb4-4d56-4362-9e1b-c4569e21b8ac", "content-length": "734", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9772026-03-24T21:00:07.162ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
9782026-03-24T21:00:07.163ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "992a1f50-96ba-4cc7-a503-1f69da30a33d", "content-length": "111", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9792026-03-24T21:00:07.163ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
9802026-03-24T21:00:07.163ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6502507-6e03-454f-b8c4-9d5aba4bc40b", "content-length": "111", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9812026-03-24T21:00:07.163ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
9822026-03-24T21:00:07.163ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3148cca-3ede-496e-b573-14e0a572c083", "content-length": "163", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9832026-03-24T21:00:07.163ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
9842026-03-24T21:00:07.164ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca1c4f4f-fafc-4505-871a-c57e1bb70413", "content-length": "163", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9852026-03-24T21:00:07.164ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
9862026-03-24T21:00:07.164ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e157fa03-b65c-4b53-81a1-791c1e23d92e", "content-length": "166", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9872026-03-24T21:00:07.164ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
9882026-03-24T21:00:07.165ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b970edec-6528-45c9-a2d9-8ae2a6f0fe80", "content-length": "166", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9892026-03-24T21:00:07.165ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
9902026-03-24T21:00:07.165ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad706985-63c0-4763-b0a9-fbf44ddbee66", "content-length": "702", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9912026-03-24T21:00:07.165ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
9922026-03-24T21:00:07.165ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "209bafe3-a994-4acb-a592-347fc7d30dd6", "content-length": "727", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9932026-03-24T21:00:07.165ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
9942026-03-24T21:00:07.165ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "acdabc3d-72ba-40ff-af9e-d80aaac8b039", "content-length": "729", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9952026-03-24T21:00:07.165ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
9962026-03-24T21:00:07.166ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ecb620d-eb42-482b-ab20-eb9757ce37f2", "content-length": "728", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9972026-03-24T21:00:07.166ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
9982026-03-24T21:00:07.166ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb1312c1-0662-4a3f-9337-dec2c009c6a0", "content-length": "722", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
9992026-03-24T21:00:07.166ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
10002026-03-24T21:00:07.166ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "39983742-f29a-4f29-b426-fab9d514f726", "content-length": "10", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10012026-03-24T21:00:07.166ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
10022026-03-24T21:00:07.166ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "63d6b3c4-633d-4768-b4cf-dbcf9bd7a2fb", "content-length": "144", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10032026-03-24T21:00:07.166ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
10042026-03-24T21:00:07.167ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fa4c5d1-8353-4648-8563-e1704e2bc47c", "content-length": "147", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10052026-03-24T21:00:07.167ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
10062026-03-24T21:00:07.167ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1072abb8-077e-416b-96a4-8458e92691ad", "content-length": "109", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10072026-03-24T21:00:07.167ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
10082026-03-24T21:00:07.167ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "50d5a85e-2d64-43b6-92b7-a30b2811dc7a", "content-length": "109", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10092026-03-24T21:00:07.167ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
10102026-03-24T21:00:07.168ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9d355ba-4289-4328-98d3-48be819d3c1c", "content-length": "163", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10112026-03-24T21:00:07.168ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
10122026-03-24T21:00:07.168ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5a6ccd1-7cbe-4dde-a26f-371ba4184212", "content-length": "163", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10132026-03-24T21:00:07.168ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
10142026-03-24T21:00:07.169ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f820b735-b22c-4e48-8a89-ca71719b3a5a", "content-length": "166", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10152026-03-24T21:00:07.169ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
10162026-03-24T21:00:07.169ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "44b0b2d0-b396-42c1-9046-57c34db783cb", "content-length": "166", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10172026-03-24T21:00:07.169ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
10182026-03-24T21:00:07.169ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "39f2f02a-fde0-4160-8495-95f11b8263d9", "content-length": "702", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10192026-03-24T21:00:07.169ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
10202026-03-24T21:00:07.170ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "45d7c87a-7382-4729-8c73-e48c47681ee7", "content-length": "727", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10212026-03-24T21:00:07.170ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
10222026-03-24T21:00:07.170ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7a44e7e-09ed-4ee9-9ebb-4082142aed8e", "content-length": "729", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10232026-03-24T21:00:07.170ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
10242026-03-24T21:00:07.170ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "44bda6a8-ecdf-4509-b78e-5ba50257949b", "content-length": "728", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10252026-03-24T21:00:07.170ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10262026-03-24T21:00:07.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
10272026-03-24T21:00:07.172ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10282026-03-24T21:00:07.175ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
10292026-03-24T21:00:07.175ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10302026-03-24T21:00:07.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
10312026-03-24T21:00:07.176ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10322026-03-24T21:00:07.178ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
10332026-03-24T21:00:07.178ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10342026-03-24T21:00:07.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
10352026-03-24T21:00:07.182ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
10362026-03-24T21:00:07.182ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
10372026-03-24T21:00:07.239ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ad6ebee-14f4-45d6-be06-7b533208478b", "content-length": "110670", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10382026-03-24T21:00:07.240ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
10392026-03-24T21:00:07.240ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
10402026-03-24T21:00:07.240ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
10412026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
10422026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
10432026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
10442026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
10452026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
10462026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bad01e0-94ae-4b81-9c56-a97adb2e15fa", "content-length": "64", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10472026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
10482026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "08772647-a223-41e0-9ccf-b88713f66523", "content-length": "64", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10492026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0d0a203-52d9-4678-bbc8-2a92f7f19a15", "content-length": "64", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10502026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1d0850e-f7e9-4b59-a692-c54500b765db", "content-length": "64", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10512026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbb6e7ea-edae-4294-98a2-f37e32da02d2", "content-length": "64", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10522026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
10532026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
10542026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
10552026-03-24T21:00:07.240ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
10562026-03-24T21:00:07.249ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "969cfabf-21e6-41b6-8d52-aad862995197", "content-length": "380540", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10572026-03-24T21:00:07.249ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7c136fe-05d5-40a2-aa27-38a69507539c", "content-length": "417734", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10582026-03-24T21:00:07.249ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "15b0142d-f7ec-4124-bf0e-b954d427d2d3", "content-length": "386101", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10592026-03-24T21:00:07.250ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "52b1de12-abce-470d-a1fd-5864aaba1ce1", "content-length": "390876", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10602026-03-24T21:00:07.255ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "beec642b-8c2c-4277-9acc-1a6174f584bc", "content-length": "383320", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10612026-03-24T21:00:07.258ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
10622026-03-24T21:00:07.258ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
10632026-03-24T21:00:07.258ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
10642026-03-24T21:00:07.258ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
10652026-03-24T21:00:07.258ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
10662026-03-24T21:00:07.259ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 (omicron_zone)
10672026-03-24T21:00:07.259ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
10682026-03-24T21:00:07.262ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "df3c6c5b-fe5e-4b5b-a385-b7b249f90bd8", "content-length": "126", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10692026-03-24T21:00:07.262ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
10702026-03-24T21:00:07.265ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
10712026-03-24T21:00:07.265ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
10722026-03-24T21:00:07.265ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a48bc53d-9626-4f7c-b9ee-9e25837425a0", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10732026-03-24T21:00:07.265ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
10742026-03-24T21:00:07.265ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
10752026-03-24T21:00:07.265ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
10762026-03-24T21:00:07.266ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7161a35a-7dea-4f9d-a936-894f46142c68", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10772026-03-24T21:00:07.266ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
10782026-03-24T21:00:07.266ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
10792026-03-24T21:00:07.266ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
10802026-03-24T21:00:07.266ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9926544-ce3a-460c-97cd-2966a086c1c9", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:00:07 GMT"} })
10812026-03-24T21:00:07.266ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
10822026-03-24T21:00:07.266ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
10832026-03-24T21:00:07.266ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
10842026-03-24T21:00:07.267ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10852026-03-24T21:00:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
10862026-03-24T21:00:07.269ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10872026-03-24T21:00:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
10882026-03-24T21:00:07.271ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10892026-03-24T21:00:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
10902026-03-24T21:00:07.365ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 91efef4f-633d-4073-a5f1-2a6928f85757 file = nexus/db-queries/src/db/datastore/inventory.rs:1757
10912026-03-24T21:00:07.365ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 91efef4f-633d-4073-a5f1-2a6928f85757 time_started = 2026-03-24 21:00:07.161292 UTC
10922026-03-24T21:00:07.365ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 19 reason = Dependency
10932026-03-24T21:00:07.365ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
10942026-03-24T21:00:07.366ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.020911589s iteration = 6 status = {"collection_id":"91efef4f-633d-4073-a5f1-2a6928f85757","time_done":"2026-03-24 21:00:07.266977 UTC","time_started":"2026-03-24 21:00:07.161292 UTC"}
10952026-03-24T21:00:07.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
10962026-03-24T21:00:07.368ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10972026-03-24T21:00:07.371ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
10982026-03-24T21:00:07.371ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
10992026-03-24T21:00:07.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
11002026-03-24T21:00:07.540ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
11012026-03-24T21:00:07.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
11022026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Dependency
11032026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 210.433434ms iteration = 19 status = {"Loaded":{"collection_id":"e0be8d7c-c2e1-4aab-8e77-8dbb701a026c","time_loaded":"2026-03-24T21:00:07.365968875Z","time_started":"2026-03-24T21:00:07.225426Z"}}
11042026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 8 reason = Dependency
11052026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 11 reason = Dependency
11062026-03-24T21:00:07.576ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
11072026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11082026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 8.75\u{b5}s iteration = 11 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
11092026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 13 reason = Dependency
11102026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11112026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11122026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11132026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11142026-03-24T21:00:07.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11152026-03-24T21:00:07.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11162026-03-24T21:00:07.578ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11172026-03-24T21:00:07.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11182026-03-24T21:00:07.581ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11192026-03-24T21:00:07.581ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11202026-03-24T21:00:07.581ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
11212026-03-24T21:00:07.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11222026-03-24T21:00:07.581ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
11232026-03-24T21:00:07.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11242026-03-24T21:00:07.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11252026-03-24T21:00:07.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11262026-03-24T21:00:07.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11272026-03-24T21:00:07.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11282026-03-24T21:00:07.583ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11292026-03-24T21:00:07.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11302026-03-24T21:00:07.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11312026-03-24T21:00:07.585ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11322026-03-24T21:00:07.586ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
11332026-03-24T21:00:07.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11342026-03-24T21:00:07.586ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
11352026-03-24T21:00:07.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11362026-03-24T21:00:07.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11372026-03-24T21:00:07.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11382026-03-24T21:00:07.587ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11392026-03-24T21:00:07.587ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11402026-03-24T21:00:07.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11412026-03-24T21:00:07.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11422026-03-24T21:00:07.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11432026-03-24T21:00:07.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11442026-03-24T21:00:07.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11452026-03-24T21:00:07.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11462026-03-24T21:00:07.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11472026-03-24T21:00:07.593ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
11482026-03-24T21:00:07.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11492026-03-24T21:00:07.593ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
11502026-03-24T21:00:07.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11512026-03-24T21:00:07.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11522026-03-24T21:00:07.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11532026-03-24T21:00:07.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11542026-03-24T21:00:07.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11552026-03-24T21:00:07.596ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.813664ms iteration = 14 status = {"physical_disks_added":0}
11562026-03-24T21:00:07.597ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
11572026-03-24T21:00:07.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11582026-03-24T21:00:07.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11592026-03-24T21:00:07.598ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
11602026-03-24T21:00:07.598ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11612026-03-24T21:00:07.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11622026-03-24T21:00:07.599ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
11632026-03-24T21:00:07.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11642026-03-24T21:00:07.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11652026-03-24T21:00:07.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11662026-03-24T21:00:07.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11672026-03-24T21:00:07.601ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11682026-03-24T21:00:07.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11692026-03-24T21:00:07.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11702026-03-24T21:00:07.603ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
11712026-03-24T21:00:07.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11722026-03-24T21:00:07.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11732026-03-24T21:00:07.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11742026-03-24T21:00:07.605ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11752026-03-24T21:00:07.606ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
11762026-03-24T21:00:07.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11772026-03-24T21:00:07.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11782026-03-24T21:00:07.607ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
11792026-03-24T21:00:07.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11802026-03-24T21:00:07.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11812026-03-24T21:00:07.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11822026-03-24T21:00:07.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11832026-03-24T21:00:07.609ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
11842026-03-24T21:00:07.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11852026-03-24T21:00:07.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11862026-03-24T21:00:07.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11872026-03-24T21:00:07.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11882026-03-24T21:00:07.612ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
11892026-03-24T21:00:07.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11902026-03-24T21:00:07.612ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
11912026-03-24T21:00:07.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11922026-03-24T21:00:07.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11932026-03-24T21:00:07.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11942026-03-24T21:00:07.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11952026-03-24T21:00:07.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11962026-03-24T21:00:07.616ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
11972026-03-24T21:00:07.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
11982026-03-24T21:00:07.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
11992026-03-24T21:00:07.618ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
12002026-03-24T21:00:07.618ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 41.838588ms iteration = 8 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"e0be8d7c-c2e1-4aab-8e77-8dbb701a026c","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
12012026-03-24T21:00:07.618ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
12022026-03-24T21:00:07.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12032026-03-24T21:00:07.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12042026-03-24T21:00:07.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12052026-03-24T21:00:07.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12062026-03-24T21:00:07.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12072026-03-24T21:00:07.626ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
12082026-03-24T21:00:07.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12092026-03-24T21:00:07.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12102026-03-24T21:00:07.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12112026-03-24T21:00:07.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12122026-03-24T21:00:07.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12132026-03-24T21:00:07.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
12142026-03-24T21:00:07.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12152026-03-24T21:00:07.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12162026-03-24T21:00:07.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12172026-03-24T21:00:07.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12182026-03-24T21:00:07.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12192026-03-24T21:00:07.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
12202026-03-24T21:00:07.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12212026-03-24T21:00:07.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12222026-03-24T21:00:07.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12232026-03-24T21:00:07.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12242026-03-24T21:00:07.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12252026-03-24T21:00:07.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12262026-03-24T21:00:07.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12272026-03-24T21:00:07.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
12282026-03-24T21:00:07.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12292026-03-24T21:00:07.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12302026-03-24T21:00:07.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12312026-03-24T21:00:07.665ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
12322026-03-24T21:00:07.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12332026-03-24T21:00:07.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12342026-03-24T21:00:07.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12352026-03-24T21:00:07.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12362026-03-24T21:00:07.669ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12372026-03-24T21:00:07.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12382026-03-24T21:00:07.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12392026-03-24T21:00:07.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
12402026-03-24T21:00:07.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12412026-03-24T21:00:07.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12422026-03-24T21:00:07.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12432026-03-24T21:00:07.678ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
12442026-03-24T21:00:07.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12452026-03-24T21:00:07.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12462026-03-24T21:00:07.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12472026-03-24T21:00:07.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12482026-03-24T21:00:07.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12492026-03-24T21:00:07.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12502026-03-24T21:00:07.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12512026-03-24T21:00:07.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
12522026-03-24T21:00:07.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12532026-03-24T21:00:07.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12542026-03-24T21:00:07.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12552026-03-24T21:00:07.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12562026-03-24T21:00:07.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12572026-03-24T21:00:07.694ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
12582026-03-24T21:00:07.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12592026-03-24T21:00:07.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12602026-03-24T21:00:07.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12612026-03-24T21:00:07.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12622026-03-24T21:00:07.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12632026-03-24T21:00:07.703ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
12642026-03-24T21:00:07.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12652026-03-24T21:00:07.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12662026-03-24T21:00:07.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12672026-03-24T21:00:07.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12682026-03-24T21:00:07.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12692026-03-24T21:00:07.710ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
12702026-03-24T21:00:07.710ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12712026-03-24T21:00:07.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12722026-03-24T21:00:07.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12732026-03-24T21:00:07.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12742026-03-24T21:00:07.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12752026-03-24T21:00:07.718ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
12762026-03-24T21:00:07.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12772026-03-24T21:00:07.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12782026-03-24T21:00:07.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12792026-03-24T21:00:07.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12802026-03-24T21:00:07.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12812026-03-24T21:00:07.723ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
12822026-03-24T21:00:07.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12832026-03-24T21:00:07.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12842026-03-24T21:00:07.725ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
12852026-03-24T21:00:07.725ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
12862026-03-24T21:00:07.725ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 9d0064c5-1eb0-41f9-855e-b6226d39ec1b parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
12872026-03-24T21:00:07.725ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12882026-03-24T21:00:07.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12892026-03-24T21:00:07.727ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12902026-03-24T21:00:07.730ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
12912026-03-24T21:00:07.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12922026-03-24T21:00:07.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12932026-03-24T21:00:07.733ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
12942026-03-24T21:00:07.733ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
12952026-03-24T21:00:07.733ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 157.034155ms iteration = 13 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
12962026-03-24T21:00:08.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 653 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 9e7145fb-c97b-4201-8275-191ebdaeb9f3 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
12972026-03-24T21:00:10.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 38 reason = Timeout
12982026-03-24T21:00:10.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
12992026-03-24T21:00:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
13002026-03-24T21:00:10.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13012026-03-24T21:00:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
13022026-03-24T21:00:10.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
13032026-03-24T21:00:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
13042026-03-24T21:00:10.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
13052026-03-24T21:00:10.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.103512ms iteration = 38 status = {"config_updated":false}
13062026-03-24T21:00:15.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
13072026-03-24T21:00:15.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13082026-03-24T21:00:15.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 39 reason = Timeout
13092026-03-24T21:00:15.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
13102026-03-24T21:00:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
13112026-03-24T21:00:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
13122026-03-24T21:00:15.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13132026-03-24T21:00:15.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13142026-03-24T21:00:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13152026-03-24T21:00:15.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13162026-03-24T21:00:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
13172026-03-24T21:00:15.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
13182026-03-24T21:00:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
13192026-03-24T21:00:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
13202026-03-24T21:00:15.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13212026-03-24T21:00:15.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
13222026-03-24T21:00:15.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
13232026-03-24T21:00:15.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.77635ms iteration = 39 status = {"config_updated":false}
13242026-03-24T21:00:15.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13252026-03-24T21:00:15.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
13262026-03-24T21:00:15.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13272026-03-24T21:00:15.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
13282026-03-24T21:00:15.555ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
13292026-03-24T21:00:15.555ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13302026-03-24T21:00:15.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13312026-03-24T21:00:15.556ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
13322026-03-24T21:00:15.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
13332026-03-24T21:00:15.558ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.142885ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
13342026-03-24T21:00:15.562ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
13352026-03-24T21:00:15.562ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.728505ms iteration = 20 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
13362026-03-24T21:00:18.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 435 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 6de74853-e2b4-44a2-ae8b-a61c4e3367ca response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
13372026-03-24T21:00:20.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 20 reason = Timeout
13382026-03-24T21:00:20.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 40 reason = Timeout
13392026-03-24T21:00:20.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
13402026-03-24T21:00:20.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
13412026-03-24T21:00:20.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
13422026-03-24T21:00:20.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
13432026-03-24T21:00:20.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13442026-03-24T21:00:20.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13452026-03-24T21:00:20.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
13462026-03-24T21:00:20.537ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
13472026-03-24T21:00:20.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
13482026-03-24T21:00:20.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
13492026-03-24T21:00:20.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
13502026-03-24T21:00:20.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
13512026-03-24T21:00:20.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
13522026-03-24T21:00:20.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.804713ms iteration = 40 status = {"config_updated":false}
13532026-03-24T21:00:20.539ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = e0be8d7c-c2e1-4aab-8e77-8dbb701a026c original_time_started = 2026-03-24 21:00:07.225426 UTC
13542026-03-24T21:00:20.539ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.000729ms iteration = 20 status = {"Loaded":{"collection_id":"e0be8d7c-c2e1-4aab-8e77-8dbb701a026c","time_loaded":"2026-03-24T21:00:20.532498356Z","time_started":"2026-03-24T21:00:07.225426Z"}}
13552026-03-24T21:00:20.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 14 reason = Timeout
13562026-03-24T21:00:20.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 14 reason = Timeout
13572026-03-24T21:00:20.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
13582026-03-24T21:00:20.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
13592026-03-24T21:00:20.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
13602026-03-24T21:00:20.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
13612026-03-24T21:00:20.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
13622026-03-24T21:00:20.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
13632026-03-24T21:00:20.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13642026-03-24T21:00:20.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13652026-03-24T21:00:20.579ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
13662026-03-24T21:00:20.579ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
13672026-03-24T21:00:20.579ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
13682026-03-24T21:00:20.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
13692026-03-24T21:00:20.580ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.323463ms iteration = 14 status = "NoSitrep"
13702026-03-24T21:00:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
13712026-03-24T21:00:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
13722026-03-24T21:00:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
13732026-03-24T21:00:20.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
13742026-03-24T21:00:20.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13752026-03-24T21:00:20.585ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
13762026-03-24T21:00:20.588ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
13772026-03-24T21:00:20.588ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
13782026-03-24T21:00:20.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.700063ms iteration = 14 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
13792026-03-24T21:00:25.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
13802026-03-24T21:00:25.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 41 reason = Timeout
13812026-03-24T21:00:25.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13822026-03-24T21:00:25.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
13832026-03-24T21:00:25.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
13842026-03-24T21:00:25.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
13852026-03-24T21:00:25.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13862026-03-24T21:00:25.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13872026-03-24T21:00:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
13882026-03-24T21:00:25.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13892026-03-24T21:00:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
13902026-03-24T21:00:25.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
13912026-03-24T21:00:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
13922026-03-24T21:00:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
13932026-03-24T21:00:25.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
13942026-03-24T21:00:25.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
13952026-03-24T21:00:25.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.471251ms iteration = 41 status = {"config_updated":false}
13962026-03-24T21:00:25.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
13972026-03-24T21:00:25.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13982026-03-24T21:00:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
13992026-03-24T21:00:25.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14002026-03-24T21:00:25.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14012026-03-24T21:00:25.555ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
14022026-03-24T21:00:25.555ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14032026-03-24T21:00:25.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14042026-03-24T21:00:25.557ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14052026-03-24T21:00:25.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14062026-03-24T21:00:25.603ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 48.032205ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14072026-03-24T21:00:25.653ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
14082026-03-24T21:00:25.653ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 121.223478ms iteration = 21 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
14092026-03-24T21:00:28.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 441 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = a995f4d8-9404-40f9-9106-56b66ce0f5bb response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
14102026-03-24T21:00:30.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 42 reason = Timeout
14112026-03-24T21:00:30.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
14122026-03-24T21:00:30.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
14132026-03-24T21:00:30.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14142026-03-24T21:00:30.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
14152026-03-24T21:00:30.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
14162026-03-24T21:00:30.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
14172026-03-24T21:00:30.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
14182026-03-24T21:00:30.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.352661ms iteration = 42 status = {"config_updated":false}
14192026-03-24T21:00:35.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 8 reason = Timeout
14202026-03-24T21:00:35.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
14212026-03-24T21:00:35.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
14222026-03-24T21:00:35.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
14232026-03-24T21:00:35.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
14242026-03-24T21:00:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
14252026-03-24T21:00:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
14262026-03-24T21:00:35.524ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1021, tv_nsec: 738237851 } })
14272026-03-24T21:00:35.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1021, tv_nsec: 738631201 } })
14282026-03-24T21:00:35.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
14292026-03-24T21:00:35.526ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1021, tv_nsec: 740706397 } })
14302026-03-24T21:00:35.527ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
14312026-03-24T21:00:35.527ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 6.541257ms iteration = 8 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
14322026-03-24T21:00:35.528ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
14332026-03-24T21:00:35.528ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
14342026-03-24T21:00:35.528ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4240751f-023a-434f-a36b-20d8519ccc15", "content-length": "26", "date": "Tue, 24 Mar 2026 21:00:35 GMT"} })
14352026-03-24T21:00:35.528ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
14362026-03-24T21:00:35.528ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
14372026-03-24T21:00:35.528ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1021, tv_nsec: 742703811 } })
14382026-03-24T21:00:35.531ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 10.484543ms iteration = 8 status = {}
14392026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
14402026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
14412026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14422026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 12 reason = Timeout
14432026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14442026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
14452026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14462026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
14472026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14482026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14492026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 43 reason = Timeout
14502026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
14512026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 21 reason = Timeout
14522026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
14532026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
14542026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
14552026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14562026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14572026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
14582026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14592026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
14602026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14612026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14622026-03-24T21:00:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
14632026-03-24T21:00:35.535ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 14.754157ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
14642026-03-24T21:00:35.537ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14652026-03-24T21:00:35.537ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14662026-03-24T21:00:35.538ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14672026-03-24T21:00:35.539ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
14682026-03-24T21:00:35.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
14692026-03-24T21:00:35.539ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14702026-03-24T21:00:35.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14712026-03-24T21:00:35.541ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14722026-03-24T21:00:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
14732026-03-24T21:00:35.542ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
14742026-03-24T21:00:35.542ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14752026-03-24T21:00:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
14762026-03-24T21:00:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
14772026-03-24T21:00:35.543ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14782026-03-24T21:00:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
14792026-03-24T21:00:35.543ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14802026-03-24T21:00:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14812026-03-24T21:00:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14822026-03-24T21:00:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
14832026-03-24T21:00:35.546ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
14842026-03-24T21:00:35.546ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
14852026-03-24T21:00:35.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
14862026-03-24T21:00:35.546ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
14872026-03-24T21:00:35.546ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1021, tv_nsec: 760334226 } })
14882026-03-24T21:00:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
14892026-03-24T21:00:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
14902026-03-24T21:00:35.547ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
14912026-03-24T21:00:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14922026-03-24T21:00:35.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14932026-03-24T21:00:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
14942026-03-24T21:00:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
14952026-03-24T21:00:35.547ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
14962026-03-24T21:00:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14972026-03-24T21:00:35.547ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1021, tv_nsec: 761286012 } })
14982026-03-24T21:00:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
14992026-03-24T21:00:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
15002026-03-24T21:00:35.548ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15012026-03-24T21:00:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15022026-03-24T21:00:35.549ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
15032026-03-24T21:00:35.549ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 15.873497ms iteration = 43 status = {"config_updated":false}
15042026-03-24T21:00:35.549ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
15052026-03-24T21:00:35.549ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
15062026-03-24T21:00:35.549ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = e0be8d7c-c2e1-4aab-8e77-8dbb701a026c original_time_started = 2026-03-24 21:00:07.225426 UTC
15072026-03-24T21:00:35.549ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 16.491385ms iteration = 21 status = {"Loaded":{"collection_id":"e0be8d7c-c2e1-4aab-8e77-8dbb701a026c","time_loaded":"2026-03-24T21:00:35.533295718Z","time_started":"2026-03-24T21:00:07.225426Z"}}
15082026-03-24T21:00:35.549ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6081bef7-ea52-40bf-a26c-a28fc1e345bb", "content-length": "26", "date": "Tue, 24 Mar 2026 21:00:35 GMT"} })
15092026-03-24T21:00:35.550ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
15102026-03-24T21:00:35.550ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
15112026-03-24T21:00:35.550ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 17.313386ms iteration = 8 status = 0
15122026-03-24T21:00:35.552ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15132026-03-24T21:00:35.552ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1021, tv_nsec: 766019350 } })
15142026-03-24T21:00:35.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15152026-03-24T21:00:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
15162026-03-24T21:00:35.554ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1021, tv_nsec: 768115746 } })
15172026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 8 reason = Timeout
15182026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
15192026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
15202026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15212026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
15222026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
15232026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 8 reason = Timeout
15242026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15252026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
15262026-03-24T21:00:35.554ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
15272026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
15282026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15292026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15302026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
15312026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
15322026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15332026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
15342026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15352026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15362026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15372026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
15382026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15392026-03-24T21:00:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
15402026-03-24T21:00:35.555ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15412026-03-24T21:00:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
15422026-03-24T21:00:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15432026-03-24T21:00:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15442026-03-24T21:00:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15452026-03-24T21:00:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
15462026-03-24T21:00:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
15472026-03-24T21:00:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15482026-03-24T21:00:35.556ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15492026-03-24T21:00:35.556ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15502026-03-24T21:00:35.556ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15512026-03-24T21:00:35.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15522026-03-24T21:00:35.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15532026-03-24T21:00:35.557ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15542026-03-24T21:00:35.558ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 8 reason = Timeout
15552026-03-24T21:00:35.558ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
15562026-03-24T21:00:35.558ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15572026-03-24T21:00:35.558ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15582026-03-24T21:00:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
15592026-03-24T21:00:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
15602026-03-24T21:00:35.559ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
15612026-03-24T21:00:35.559ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
15622026-03-24T21:00:35.559ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15632026-03-24T21:00:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
15642026-03-24T21:00:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
15652026-03-24T21:00:35.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15662026-03-24T21:00:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
15672026-03-24T21:00:35.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15682026-03-24T21:00:35.561ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15692026-03-24T21:00:35.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15702026-03-24T21:00:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15712026-03-24T21:00:35.564ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15722026-03-24T21:00:35.564ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 9.426554ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
15732026-03-24T21:00:35.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15742026-03-24T21:00:35.564ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 17.829701ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
15752026-03-24T21:00:35.564ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15762026-03-24T21:00:35.564ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
15772026-03-24T21:00:35.564ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15782026-03-24T21:00:35.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
15792026-03-24T21:00:35.566ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15802026-03-24T21:00:35.566ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15812026-03-24T21:00:35.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15822026-03-24T21:00:35.568ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15832026-03-24T21:00:35.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15842026-03-24T21:00:35.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15852026-03-24T21:00:35.570ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
15862026-03-24T21:00:35.570ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15872026-03-24T21:00:35.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15882026-03-24T21:00:35.573ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
15892026-03-24T21:00:35.573ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 19.267149ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
15902026-03-24T21:00:35.575ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 8 reason = Timeout
15912026-03-24T21:00:35.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 15 reason = Timeout
15922026-03-24T21:00:35.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 15 reason = Timeout
15932026-03-24T21:00:35.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
15942026-03-24T21:00:35.575ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 18.13\u{b5}s iteration = 8 status = {"disabled":true,"errors":[],"sps":[]}
15952026-03-24T21:00:35.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
15962026-03-24T21:00:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
15972026-03-24T21:00:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
15982026-03-24T21:00:35.576ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
15992026-03-24T21:00:35.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
16002026-03-24T21:00:35.577ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
16012026-03-24T21:00:35.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
16022026-03-24T21:00:35.577ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bebcbff-5704-47e3-b800-fb7607b5f536", "content-length": "372", "date": "Tue, 24 Mar 2026 21:00:35 GMT"} })
16032026-03-24T21:00:35.577ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16042026-03-24T21:00:35.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16052026-03-24T21:00:35.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16062026-03-24T21:00:35.579ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16072026-03-24T21:00:35.582ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
16082026-03-24T21:00:35.582ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
16092026-03-24T21:00:35.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
16102026-03-24T21:00:35.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
16112026-03-24T21:00:35.584ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 9.355482ms iteration = 15 status = "NoSitrep"
16122026-03-24T21:00:35.587ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
16132026-03-24T21:00:35.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
16142026-03-24T21:00:35.589ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
16152026-03-24T21:00:35.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
16162026-03-24T21:00:35.594ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16172026-03-24T21:00:35.596ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
16182026-03-24T21:00:35.598ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
16192026-03-24T21:00:35.598ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
16202026-03-24T21:00:35.599ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
16212026-03-24T21:00:35.599ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
16222026-03-24T21:00:35.599ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
16232026-03-24T21:00:35.599ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
16242026-03-24T21:00:35.599ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
16252026-03-24T21:00:35.599ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16262026-03-24T21:00:35.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16272026-03-24T21:00:35.602ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
16282026-03-24T21:00:35.602ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
16292026-03-24T21:00:35.602ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
16302026-03-24T21:00:35.602ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
16312026-03-24T21:00:35.602ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 27.151192ms iteration = 15 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
16322026-03-24T21:00:35.603ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
16332026-03-24T21:00:35.603ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 70.079819ms iteration = 22 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
16342026-03-24T21:00:35.604ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
16352026-03-24T21:00:35.604ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
16362026-03-24T21:00:35.614ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "16996712-997e-4d58-ab88-83736fd68984", "date": "Tue, 24 Mar 2026 21:00:35 GMT"} })
16372026-03-24T21:00:35.614ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16382026-03-24T21:00:35.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16392026-03-24T21:00:35.618ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16402026-03-24T21:00:35.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16412026-03-24T21:00:35.622ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16422026-03-24T21:00:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16432026-03-24T21:00:35.627ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
16442026-03-24T21:00:35.627ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
16452026-03-24T21:00:35.628ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "6dd4b81f-2870-4bfa-a9e4-3e52174e3124", "date": "Tue, 24 Mar 2026 21:00:35 GMT"} })
16462026-03-24T21:00:35.628ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 74.087118ms iteration = 12 status = {}
16472026-03-24T21:00:35.629ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
16482026-03-24T21:00:35.629ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 96.759539ms iteration = 12 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
16492026-03-24T21:00:35.632ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16502026-03-24T21:00:35.632ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
16512026-03-24T21:00:35.632ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16522026-03-24T21:00:35.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
16532026-03-24T21:00:35.642ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16542026-03-24T21:00:35.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16552026-03-24T21:00:35.643ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 89.074871ms iteration = 8 status = {"errors":[],"finish_invoked_ok":[]}
16562026-03-24T21:00:35.646ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16572026-03-24T21:00:35.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16582026-03-24T21:00:35.654ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 99.923974ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16592026-03-24T21:00:35.655ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 97.462308ms iteration = 8 status = {"errors":[],"requests_created_ok":[]}
16602026-03-24T21:00:35.659ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16612026-03-24T21:00:35.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
16622026-03-24T21:00:35.661ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
16632026-03-24T21:00:35.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
16642026-03-24T21:00:35.662ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16652026-03-24T21:00:35.663ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 109.281516ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16662026-03-24T21:00:35.665ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16672026-03-24T21:00:35.665ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16682026-03-24T21:00:35.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
16692026-03-24T21:00:35.669ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16702026-03-24T21:00:35.672ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
16712026-03-24T21:00:35.672ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16722026-03-24T21:00:35.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
16732026-03-24T21:00:35.676ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
16742026-03-24T21:00:35.677ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7e12b0a-0fed-470b-a368-d9f8e961690f", "content-length": "691", "date": "Tue, 24 Mar 2026 21:00:35 GMT"} })
16752026-03-24T21:00:35.677ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
16762026-03-24T21:00:35.677ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
16772026-03-24T21:00:35.677ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
16782026-03-24T21:00:35.677ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 144.045074ms iteration = 8 status = {}
16792026-03-24T21:00:35.682ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 148.901175ms iteration = 15 status = {"physical_disks_added":0}
16802026-03-24T21:00:35.699ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16812026-03-24T21:00:35.699ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
16822026-03-24T21:00:35.699ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 145.121253ms iteration = 8 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
16832026-03-24T21:00:35.706ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 152.146693ms iteration = 8 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
16842026-03-24T21:00:35.781ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "332e2378-55c9-461b-b9de-24c83deda38b", "date": "Tue, 24 Mar 2026 21:00:35 GMT"} })
16852026-03-24T21:00:35.781ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
16862026-03-24T21:00:35.781ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16872026-03-24T21:00:35.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16882026-03-24T21:00:35.841ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16892026-03-24T21:00:35.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16902026-03-24T21:00:35.845ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16912026-03-24T21:00:35.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16922026-03-24T21:00:35.847ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
16932026-03-24T21:00:35.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16942026-03-24T21:00:35.850ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 317.122101ms iteration = 8 status = {}
16952026-03-24T21:00:36.166ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 1
16962026-03-24T21:00:36.166ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221
16972026-03-24T21:00:36.167ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
16982026-03-24T21:00:36.167ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:34593
16992026-03-24T21:00:36.167ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:34593 req_id = 573a4583-5891-4431-839f-92a9b0fca721 uri = /metrics/producers version = 1.0.0
17002026-03-24T21:00:36.167ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:34593 req_id = 573a4583-5891-4431-839f-92a9b0fca721 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
17012026-03-24T21:00:36.167ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:34593 req_id = 573a4583-5891-4431-839f-92a9b0fca721 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
17022026-03-24T21:00:36.216ZINFOnexus: assigned collector to new producer collector_id = a309df7c-2a20-4201-8e80-eb9f7e355177 file = nexus/src/app/oximeter.rs:76 producer_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb
17032026-03-24T21:00:36.216ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 48829 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:34593 req_id = 573a4583-5891-4431-839f-92a9b0fca721 response_code = 201 uri = /metrics/producers
17042026-03-24T21:00:36.216ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "573a4583-5891-4431-839f-92a9b0fca721", "content-length": "41", "date": "Tue, 24 Mar 2026 21:00:36 GMT"} })
17052026-03-24T21:00:36.216ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
17062026-03-24T21:00:38.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 363 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 208d7b5a-548c-48e1-b6e4-e97ae5feeb6e response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
17072026-03-24T21:00:40.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 44 reason = Timeout
17082026-03-24T21:00:40.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
17092026-03-24T21:00:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
17102026-03-24T21:00:40.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17112026-03-24T21:00:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
17122026-03-24T21:00:40.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
17132026-03-24T21:00:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
17142026-03-24T21:00:40.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
17152026-03-24T21:00:40.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.790984ms iteration = 44 status = {"config_updated":false}
17162026-03-24T21:00:45.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
17172026-03-24T21:00:45.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 45 reason = Timeout
17182026-03-24T21:00:45.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17192026-03-24T21:00:45.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
17202026-03-24T21:00:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
17212026-03-24T21:00:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17222026-03-24T21:00:45.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17232026-03-24T21:00:45.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17242026-03-24T21:00:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17252026-03-24T21:00:45.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17262026-03-24T21:00:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17272026-03-24T21:00:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
17282026-03-24T21:00:45.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
17292026-03-24T21:00:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
17302026-03-24T21:00:45.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17312026-03-24T21:00:45.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17322026-03-24T21:00:45.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
17332026-03-24T21:00:45.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.891163ms iteration = 45 status = {"config_updated":false}
17342026-03-24T21:00:45.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17352026-03-24T21:00:45.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
17362026-03-24T21:00:45.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17372026-03-24T21:00:45.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17382026-03-24T21:00:45.555ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
17392026-03-24T21:00:45.555ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17402026-03-24T21:00:45.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17412026-03-24T21:00:45.556ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17422026-03-24T21:00:45.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17432026-03-24T21:00:45.557ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
17442026-03-24T21:00:45.557ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.787339ms iteration = 23 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
17452026-03-24T21:00:45.558ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.253948ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
17462026-03-24T21:00:48.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 591 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = feb9f513-9719-47d6-8229-c4fe444f5787 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
17472026-03-24T21:00:50.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 22 reason = Timeout
17482026-03-24T21:00:50.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 46 reason = Timeout
17492026-03-24T21:00:50.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
17502026-03-24T21:00:50.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
17512026-03-24T21:00:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
17522026-03-24T21:00:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
17532026-03-24T21:00:50.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17542026-03-24T21:00:50.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17552026-03-24T21:00:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
17562026-03-24T21:00:50.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
17572026-03-24T21:00:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
17582026-03-24T21:00:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
17592026-03-24T21:00:50.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
17602026-03-24T21:00:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
17612026-03-24T21:00:50.540ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = e0be8d7c-c2e1-4aab-8e77-8dbb701a026c original_time_started = 2026-03-24 21:00:07.225426 UTC
17622026-03-24T21:00:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.932133ms iteration = 22 status = {"Loaded":{"collection_id":"e0be8d7c-c2e1-4aab-8e77-8dbb701a026c","time_loaded":"2026-03-24T21:00:50.533067719Z","time_started":"2026-03-24T21:00:07.225426Z"}}
17632026-03-24T21:00:50.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
17642026-03-24T21:00:50.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.999136ms iteration = 46 status = {"config_updated":false}
17652026-03-24T21:00:50.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 16 reason = Timeout
17662026-03-24T21:00:50.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 16 reason = Timeout
17672026-03-24T21:00:50.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
17682026-03-24T21:00:50.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
17692026-03-24T21:00:50.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
17702026-03-24T21:00:50.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
17712026-03-24T21:00:50.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
17722026-03-24T21:00:50.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
17732026-03-24T21:00:50.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17742026-03-24T21:00:50.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17752026-03-24T21:00:50.580ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
17762026-03-24T21:00:50.580ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
17772026-03-24T21:00:50.580ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
17782026-03-24T21:00:50.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
17792026-03-24T21:00:50.582ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.29847ms iteration = 16 status = "NoSitrep"
17802026-03-24T21:00:50.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
17812026-03-24T21:00:50.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
17822026-03-24T21:00:50.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
17832026-03-24T21:00:50.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
17842026-03-24T21:00:50.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17852026-03-24T21:00:50.586ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
17862026-03-24T21:00:50.588ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
17872026-03-24T21:00:50.588ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
17882026-03-24T21:00:50.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.797135ms iteration = 16 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
17892026-03-24T21:00:55.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
17902026-03-24T21:00:55.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 47 reason = Timeout
17912026-03-24T21:00:55.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
17922026-03-24T21:00:55.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17932026-03-24T21:00:55.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
17942026-03-24T21:00:55.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17952026-03-24T21:00:55.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17962026-03-24T21:00:55.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17972026-03-24T21:00:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17982026-03-24T21:00:55.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17992026-03-24T21:00:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
18002026-03-24T21:00:55.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18012026-03-24T21:00:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18022026-03-24T21:00:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18032026-03-24T21:00:55.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18042026-03-24T21:00:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18052026-03-24T21:00:55.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
18062026-03-24T21:00:55.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.781889ms iteration = 47 status = {"config_updated":false}
18072026-03-24T21:00:55.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18082026-03-24T21:00:55.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
18092026-03-24T21:00:55.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18102026-03-24T21:00:55.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18112026-03-24T21:00:55.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
18122026-03-24T21:00:55.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18132026-03-24T21:00:55.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18142026-03-24T21:00:55.555ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18152026-03-24T21:00:55.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18162026-03-24T21:00:55.560ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.505938ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18172026-03-24T21:00:55.563ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
18182026-03-24T21:00:55.563ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.840832ms iteration = 24 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
18192026-03-24T21:00:58.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 358 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 7335e527-a901-4171-a4bc-f8cd18c67f9d response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
18202026-03-24T21:01:00.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 48 reason = Timeout
18212026-03-24T21:01:00.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18222026-03-24T21:01:00.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18232026-03-24T21:01:00.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18242026-03-24T21:01:00.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
18252026-03-24T21:01:00.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18262026-03-24T21:01:00.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18272026-03-24T21:01:00.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
18282026-03-24T21:01:00.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.201924ms iteration = 48 status = {"config_updated":false}
18292026-03-24T21:01:05.515ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
18302026-03-24T21:01:05.516ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
18312026-03-24T21:01:05.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18322026-03-24T21:01:05.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18332026-03-24T21:01:05.517ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18342026-03-24T21:01:05.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18352026-03-24T21:01:05.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18362026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
18372026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
18382026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 9 reason = Timeout
18392026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18402026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
18412026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
18422026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
18432026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
18442026-03-24T21:01:05.520ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2026-03-24 20:51:05.520304242 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
18452026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
18462026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
18472026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
18482026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
18492026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
18502026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
18512026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
18522026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18532026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
18542026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
18552026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
18562026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
18572026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18582026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
18592026-03-24T21:01:05.520ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
18602026-03-24T21:01:05.520ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
18612026-03-24T21:01:05.521ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = ExternalNetworking step id = Ensure
18622026-03-24T21:01:05.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
18632026-03-24T21:01:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("93b6e989-1f7d-4884-aec9-ecbc076ed6c4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 735359853 } })
18642026-03-24T21:01:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("27f3a9c5-67c7-41c5-994d-9db513cda557.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2f7ea2ed-fd45-4cd7-9fb1-f48119541118.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 735389794 } })
18652026-03-24T21:01:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
18662026-03-24T21:01:05.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18672026-03-24T21:01:05.521ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
18682026-03-24T21:01:05.521ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.562872ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
18692026-03-24T21:01:05.521ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
18702026-03-24T21:01:05.521ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
18712026-03-24T21:01:05.521ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.666765ms iteration = 5 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
18722026-03-24T21:01:05.521ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
18732026-03-24T21:01:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18742026-03-24T21:01:05.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
18752026-03-24T21:01:05.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
18762026-03-24T21:01:05.523ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "0a501a73-99ce-4701-9daa-b6a985083658", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
18772026-03-24T21:01:05.523ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
18782026-03-24T21:01:05.523ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "3722eb4b-cd2f-4700-9020-2d30e6af73ef", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
18792026-03-24T21:01:05.523ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
18802026-03-24T21:01:05.524ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
18812026-03-24T21:01:05.524ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 738281542 } })
18822026-03-24T21:01:05.524ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 3.752331ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
18832026-03-24T21:01:05.524ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2026-03-24 20:51:05.520304242 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
18842026-03-24T21:01:05.524ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.875274ms iteration = 5 status = {"expiration":"2026-03-24T20:51:05.520304242Z","pruned":[]}
18852026-03-24T21:01:05.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
18862026-03-24T21:01:05.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18872026-03-24T21:01:05.525ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
18882026-03-24T21:01:05.525ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18892026-03-24T21:01:05.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
18902026-03-24T21:01:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "e9e6b03d-3bfa-4479-8610-c9c3cc4a405a", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
18912026-03-24T21:01:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
18922026-03-24T21:01:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "70f243ee-cbd8-49fd-acfa-c7bd4798a10d", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
18932026-03-24T21:01:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
18942026-03-24T21:01:05.526ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
18952026-03-24T21:01:05.526ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 5.987222ms iteration = 9 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
18962026-03-24T21:01:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "5dff3d38-6645-41a6-8562-ea1cea306326", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
18972026-03-24T21:01:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
18982026-03-24T21:01:05.526ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
18992026-03-24T21:01:05.526ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 6.153995ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
19002026-03-24T21:01:05.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19012026-03-24T21:01:05.527ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.355169ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
19022026-03-24T21:01:05.528ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 742373252 } })
19032026-03-24T21:01:05.528ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
19042026-03-24T21:01:05.528ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19052026-03-24T21:01:05.528ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19062026-03-24T21:01:05.528ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 743173314 } })
19072026-03-24T21:01:05.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
19082026-03-24T21:01:05.530ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
19092026-03-24T21:01:05.530ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
19102026-03-24T21:01:05.531ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
19112026-03-24T21:01:05.531ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
19122026-03-24T21:01:05.531ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
19132026-03-24T21:01:05.531ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19142026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19152026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Timeout
19162026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19172026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 23 reason = Timeout
19182026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
19192026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
19202026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19212026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
19222026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19232026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 49 reason = Timeout
19242026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19252026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
19262026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19272026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 14 reason = Timeout
19282026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19292026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 13 reason = Timeout
19302026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19312026-03-24T21:01:05.532ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "09185bc3-3e4d-4990-bfea-22c279565396", "content-length": "26", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} })
19322026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
19332026-03-24T21:01:05.532ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
19342026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19352026-03-24T21:01:05.532ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
19362026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19372026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
19382026-03-24T21:01:05.532ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
19392026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
19402026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19412026-03-24T21:01:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19422026-03-24T21:01:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19432026-03-24T21:01:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19442026-03-24T21:01:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
19452026-03-24T21:01:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19462026-03-24T21:01:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19472026-03-24T21:01:05.533ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 747481140 } })
19482026-03-24T21:01:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
19492026-03-24T21:01:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
19502026-03-24T21:01:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19512026-03-24T21:01:05.536ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19522026-03-24T21:01:05.537ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
19532026-03-24T21:01:05.537ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
19542026-03-24T21:01:05.537ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
19552026-03-24T21:01:05.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19562026-03-24T21:01:05.537ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19572026-03-24T21:01:05.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19582026-03-24T21:01:05.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19592026-03-24T21:01:05.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19602026-03-24T21:01:05.537ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 17.363558ms iteration = 9 status = {}
19612026-03-24T21:01:05.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19622026-03-24T21:01:05.539ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19632026-03-24T21:01:05.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
19642026-03-24T21:01:05.539ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19652026-03-24T21:01:05.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19662026-03-24T21:01:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
19672026-03-24T21:01:05.541ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
19682026-03-24T21:01:05.541ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19692026-03-24T21:01:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
19702026-03-24T21:01:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
19712026-03-24T21:01:05.541ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19722026-03-24T21:01:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19732026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19742026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19752026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19762026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19772026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
19782026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19792026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
19802026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
19812026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19822026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19832026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
19842026-03-24T21:01:05.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19852026-03-24T21:01:05.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19862026-03-24T21:01:05.545ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
19872026-03-24T21:01:05.545ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19882026-03-24T21:01:05.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
19892026-03-24T21:01:05.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
19902026-03-24T21:01:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
19912026-03-24T21:01:05.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19922026-03-24T21:01:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
19932026-03-24T21:01:05.547ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
19942026-03-24T21:01:05.547ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19952026-03-24T21:01:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19962026-03-24T21:01:05.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
19972026-03-24T21:01:05.548ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19982026-03-24T21:01:05.548ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 762959928 } })
19992026-03-24T21:01:05.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20002026-03-24T21:01:05.549ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
20012026-03-24T21:01:05.549ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20022026-03-24T21:01:05.549ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 763714488 } })
20032026-03-24T21:01:05.549ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
20042026-03-24T21:01:05.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20052026-03-24T21:01:05.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20062026-03-24T21:01:05.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20072026-03-24T21:01:05.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
20082026-03-24T21:01:05.550ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
20092026-03-24T21:01:05.550ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
20102026-03-24T21:01:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
20112026-03-24T21:01:05.551ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
20122026-03-24T21:01:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
20132026-03-24T21:01:05.551ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20142026-03-24T21:01:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
20152026-03-24T21:01:05.551ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
20162026-03-24T21:01:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20172026-03-24T21:01:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
20182026-03-24T21:01:05.551ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "79b573d0-9d15-4653-bfa8-47840fa7f146", "content-length": "26", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} })
20192026-03-24T21:01:05.552ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
20202026-03-24T21:01:05.552ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
20212026-03-24T21:01:05.552ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20222026-03-24T21:01:05.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20232026-03-24T21:01:05.552ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20242026-03-24T21:01:05.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20252026-03-24T21:01:05.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20262026-03-24T21:01:05.552ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = e0be8d7c-c2e1-4aab-8e77-8dbb701a026c original_time_started = 2026-03-24 21:00:07.225426 UTC
20272026-03-24T21:01:05.552ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 20.552244ms iteration = 23 status = {"Loaded":{"collection_id":"e0be8d7c-c2e1-4aab-8e77-8dbb701a026c","time_loaded":"2026-03-24T21:01:05.532428559Z","time_started":"2026-03-24T21:00:07.225426Z"}}
20282026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
20292026-03-24T21:01:05.553ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
20302026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 20.314178ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
20312026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20322026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 21.025667ms iteration = 9 status = 0
20332026-03-24T21:01:05.553ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 768154908 } })
20342026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
20352026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
20362026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 9 reason = Timeout
20372026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
20382026-03-24T21:01:05.553ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
20392026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 5 reason = Timeout
20402026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20412026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
20422026-03-24T21:01:05.553ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
20432026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
20442026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
20452026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
20462026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20472026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
20482026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
20492026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
20502026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
20512026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
20522026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
20532026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20542026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
20552026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20562026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
20572026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20582026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
20592026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
20602026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
20612026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
20622026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
20632026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
20642026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
20652026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
20662026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
20672026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20682026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
20692026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20702026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20712026-03-24T21:01:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20722026-03-24T21:01:05.556ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
20732026-03-24T21:01:05.556ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 23.916945ms iteration = 49 status = {"config_updated":false}
20742026-03-24T21:01:05.556ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.463286ms iteration = 9 status = {"errors":[],"finish_invoked_ok":[]}
20752026-03-24T21:01:05.556ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 771217750 } })
20762026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 12 reason = Timeout
20772026-03-24T21:01:05.557ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
20782026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20792026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 14.64\u{b5}s iteration = 12 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
20802026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 5 reason = Timeout
20812026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
20822026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 9 reason = Timeout
20832026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
20842026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 5 reason = Timeout
20852026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
20862026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20872026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20882026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
20892026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20902026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
20912026-03-24T21:01:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
20922026-03-24T21:01:05.558ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
20932026-03-24T21:01:05.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20942026-03-24T21:01:05.558ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.05765ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
20952026-03-24T21:01:05.558ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20962026-03-24T21:01:05.558ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
20972026-03-24T21:01:05.558ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20982026-03-24T21:01:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20992026-03-24T21:01:05.558ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
21002026-03-24T21:01:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21012026-03-24T21:01:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
21022026-03-24T21:01:05.558ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21032026-03-24T21:01:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
21042026-03-24T21:01:05.559ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21052026-03-24T21:01:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21062026-03-24T21:01:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21072026-03-24T21:01:05.559ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21082026-03-24T21:01:05.559ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
21092026-03-24T21:01:05.559ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 2.818256ms iteration = 5 status = {"by_rx":{},"error":null}
21102026-03-24T21:01:05.559ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21112026-03-24T21:01:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21122026-03-24T21:01:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21132026-03-24T21:01:05.560ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
21142026-03-24T21:01:05.560ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
21152026-03-24T21:01:05.560ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 6.193817ms iteration = 5 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
21162026-03-24T21:01:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
21172026-03-24T21:01:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
21182026-03-24T21:01:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21192026-03-24T21:01:05.560ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21202026-03-24T21:01:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21212026-03-24T21:01:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
21222026-03-24T21:01:05.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21232026-03-24T21:01:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
21242026-03-24T21:01:05.562ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21252026-03-24T21:01:05.562ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21262026-03-24T21:01:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
21272026-03-24T21:01:05.562ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21282026-03-24T21:01:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21292026-03-24T21:01:05.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21302026-03-24T21:01:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
21312026-03-24T21:01:05.562ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 8.797147ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21322026-03-24T21:01:05.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21332026-03-24T21:01:05.563ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21342026-03-24T21:01:05.563ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
21352026-03-24T21:01:05.563ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21362026-03-24T21:01:05.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21372026-03-24T21:01:05.564ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
21382026-03-24T21:01:05.564ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
21392026-03-24T21:01:05.564ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21402026-03-24T21:01:05.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21412026-03-24T21:01:05.564ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
21422026-03-24T21:01:05.565ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21432026-03-24T21:01:05.566ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21442026-03-24T21:01:05.566ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 9.751923ms iteration = 9 status = {"errors":[],"requests_created_ok":[]}
21452026-03-24T21:01:05.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21462026-03-24T21:01:05.567ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
21472026-03-24T21:01:05.567ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
21482026-03-24T21:01:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
21492026-03-24T21:01:05.567ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
21502026-03-24T21:01:05.569ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21512026-03-24T21:01:05.569ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21522026-03-24T21:01:05.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21532026-03-24T21:01:05.570ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
21542026-03-24T21:01:05.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
21552026-03-24T21:01:05.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21562026-03-24T21:01:05.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21572026-03-24T21:01:05.572ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
21582026-03-24T21:01:05.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21592026-03-24T21:01:05.573ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21602026-03-24T21:01:05.573ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21612026-03-24T21:01:05.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21622026-03-24T21:01:05.574ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
21632026-03-24T21:01:05.574ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
21642026-03-24T21:01:05.575ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
21652026-03-24T21:01:05.575ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 5 reason = Timeout
21662026-03-24T21:01:05.576ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
21672026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 9 reason = Timeout
21682026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
21692026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 11.58\u{b5}s iteration = 9 status = {"disabled":true,"errors":[],"sps":[]}
21702026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): activating background_task = trust_quorum_manager iteration = 5 reason = Timeout
21712026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): activating background_task = attached_subnet_manager iteration = 5 reason = Timeout
21722026-03-24T21:01:05.576ZINFOnexus (BackgroundTasks): starting attached subnet manager work background_task = attached_subnet_manager file = nexus/src/app/background/tasks/attached_subnets.rs:374
21732026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
21742026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 17 reason = Timeout
21752026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
21762026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
21772026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
21782026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
21792026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
21802026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2026-03-24 20:57:07.125641 UTC dns_group = external generation = 2
21812026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 56.095813ms iteration = 5 status = {"generation":2}
21822026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 17 reason = Timeout
21832026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
21842026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
21852026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
21862026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21872026-03-24T21:01:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
21882026-03-24T21:01:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21892026-03-24T21:01:05.577ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1051, tv_nsec: 791539008 } })
21902026-03-24T21:01:05.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
21912026-03-24T21:01:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
21922026-03-24T21:01:05.579ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
21932026-03-24T21:01:05.579ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "77598d55-e269-4a84-af59-cb050ff51705", "content-length": "372", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} })
21942026-03-24T21:01:05.579ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21952026-03-24T21:01:05.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21962026-03-24T21:01:05.582ZDEBGnexus (BackgroundTasks): roles background_task = attached_subnet_manager roles = RoleSet { roles: {} }
21972026-03-24T21:01:05.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = attached_subnet_manager resource = Database result = Ok(())
21982026-03-24T21:01:05.592ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
21992026-03-24T21:01:05.597ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
22002026-03-24T21:01:05.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
22012026-03-24T21:01:05.598ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22022026-03-24T21:01:05.599ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22032026-03-24T21:01:05.600ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
22042026-03-24T21:01:05.602ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
22052026-03-24T21:01:05.602ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
22062026-03-24T21:01:05.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
22072026-03-24T21:01:05.602ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Fleet result = Ok(())
22082026-03-24T21:01:05.602ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
22092026-03-24T21:01:05.602ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22102026-03-24T21:01:05.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
22112026-03-24T21:01:05.606ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22122026-03-24T21:01:05.606ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
22132026-03-24T21:01:05.606ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
22142026-03-24T21:01:05.606ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
22152026-03-24T21:01:05.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
22162026-03-24T21:01:05.608ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
22172026-03-24T21:01:05.608ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
22182026-03-24T21:01:05.612ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
22192026-03-24T21:01:05.612ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
22202026-03-24T21:01:05.612ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
22212026-03-24T21:01:05.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
22222026-03-24T21:01:05.613ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
22232026-03-24T21:01:05.613ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
22242026-03-24T21:01:05.613ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
22252026-03-24T21:01:05.613ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
22262026-03-24T21:01:05.613ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
22272026-03-24T21:01:05.613ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22282026-03-24T21:01:05.613ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
22292026-03-24T21:01:05.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22302026-03-24T21:01:05.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
22312026-03-24T21:01:05.614ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
22322026-03-24T21:01:05.616ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 59.239047ms iteration = 5 status = {"dispatched":[],"errors":[],"glob_version":"242.0.0","globs_reprocessed":{},"no_receivers":[]}
22332026-03-24T21:01:05.616ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
22342026-03-24T21:01:05.617ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = SledList step id = Fetch
22352026-03-24T21:01:05.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22362026-03-24T21:01:05.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22372026-03-24T21:01:05.619ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22382026-03-24T21:01:05.619ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
22392026-03-24T21:01:05.619ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 86.894923ms iteration = 25 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
22402026-03-24T21:01:05.621ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22412026-03-24T21:01:05.622ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 46.345899ms iteration = 17 status = "NoSitrep"
22422026-03-24T21:01:05.622ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
22432026-03-24T21:01:05.622ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
22442026-03-24T21:01:05.623ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
22452026-03-24T21:01:05.623ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
22462026-03-24T21:01:05.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
22472026-03-24T21:01:05.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22482026-03-24T21:01:05.624ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
22492026-03-24T21:01:05.626ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
22502026-03-24T21:01:05.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
22512026-03-24T21:01:05.627ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
22522026-03-24T21:01:05.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22532026-03-24T21:01:05.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22542026-03-24T21:01:05.627ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
22552026-03-24T21:01:05.627ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
22562026-03-24T21:01:05.628ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "077d2546-82b5-4dea-9d9e-92200baa5a90", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} })
22572026-03-24T21:01:05.628ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22582026-03-24T21:01:05.628ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22592026-03-24T21:01:05.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22602026-03-24T21:01:05.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22612026-03-24T21:01:05.630ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
22622026-03-24T21:01:05.630ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
22632026-03-24T21:01:05.630ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 53.494573ms iteration = 17 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
22642026-03-24T21:01:05.631ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22652026-03-24T21:01:05.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22662026-03-24T21:01:05.632ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
22672026-03-24T21:01:05.633ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
22682026-03-24T21:01:05.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
22692026-03-24T21:01:05.634ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
22702026-03-24T21:01:05.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22712026-03-24T21:01:05.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22722026-03-24T21:01:05.636ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22732026-03-24T21:01:05.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22742026-03-24T21:01:05.637ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = DeployNexusRecords step id = Ensure
22752026-03-24T21:01:05.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22762026-03-24T21:01:05.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22772026-03-24T21:01:05.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22782026-03-24T21:01:05.641ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
22792026-03-24T21:01:05.641ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22802026-03-24T21:01:05.642ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
22812026-03-24T21:01:05.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22822026-03-24T21:01:05.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22832026-03-24T21:01:05.650ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "aca9bcbc-78e6-47ff-931b-babcbb3978a2", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} })
22842026-03-24T21:01:05.650ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 96.923444ms iteration = 13 status = {}
22852026-03-24T21:01:05.663ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
22862026-03-24T21:01:05.665ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "90c1df68-5469-4407-a1c3-b27dd1fe8608", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} })
22872026-03-24T21:01:05.665ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
22882026-03-24T21:01:05.665ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
22892026-03-24T21:01:05.665ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 89.288568ms iteration = 5 status = {"errors":[],"probes_by_sled":{"b4293d4f-3308-4dfa-b261-a103215913e4":0}}
22902026-03-24T21:01:05.665ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Dependency
22912026-03-24T21:01:05.665ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
22922026-03-24T21:01:05.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
22932026-03-24T21:01:05.665ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22942026-03-24T21:01:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
22952026-03-24T21:01:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22962026-03-24T21:01:05.668ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22972026-03-24T21:01:05.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22982026-03-24T21:01:05.670ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
22992026-03-24T21:01:05.670ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
23002026-03-24T21:01:05.670ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23012026-03-24T21:01:05.670ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
23022026-03-24T21:01:05.670ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 137.991731ms iteration = 13 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
23032026-03-24T21:01:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23042026-03-24T21:01:05.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
23052026-03-24T21:01:05.671ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23062026-03-24T21:01:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23072026-03-24T21:01:05.672ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
23082026-03-24T21:01:05.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23092026-03-24T21:01:05.672ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 118.418402ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
23102026-03-24T21:01:05.673ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23112026-03-24T21:01:05.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
23122026-03-24T21:01:05.674ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
23132026-03-24T21:01:05.674ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 120.426197ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
23142026-03-24T21:01:05.674ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 120.461149ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
23152026-03-24T21:01:05.676ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23162026-03-24T21:01:05.676ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
23172026-03-24T21:01:05.676ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
23182026-03-24T21:01:05.676ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23192026-03-24T21:01:05.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
23202026-03-24T21:01:05.676ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
23212026-03-24T21:01:05.676ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23222026-03-24T21:01:05.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23232026-03-24T21:01:05.677ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
23242026-03-24T21:01:05.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
23252026-03-24T21:01:05.679ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
23262026-03-24T21:01:05.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
23272026-03-24T21:01:05.680ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 126.46123ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
23282026-03-24T21:01:05.681ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2026-03-24T20:56:59.957750Z, time_modified: 2026-03-24T20:56:59.957750Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }) }
23292026-03-24T21:01:05.681ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 160.919129ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]}},"default_endpoint":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
23302026-03-24T21:01:05.681ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
23312026-03-24T21:01:05.681ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
23322026-03-24T21:01:05.681ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 135.637658ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
23332026-03-24T21:01:05.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
23342026-03-24T21:01:05.681ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
23352026-03-24T21:01:05.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
23362026-03-24T21:01:05.685ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a20f0c1e-8f06-4621-9253-4a0422f1aadd", "content-length": "372", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} })
23372026-03-24T21:01:05.685ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23382026-03-24T21:01:05.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23392026-03-24T21:01:05.685ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
23402026-03-24T21:01:05.685ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
23412026-03-24T21:01:05.685ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2026-03-24 20:57:05.047625 UTC dns_group = internal generation = 1
23422026-03-24T21:01:05.685ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 169.660495ms iteration = 5 status = {"generation":1}
23432026-03-24T21:01:05.687ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23442026-03-24T21:01:05.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
23452026-03-24T21:01:05.699ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23462026-03-24T21:01:05.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
23472026-03-24T21:01:05.705ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23482026-03-24T21:01:05.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
23492026-03-24T21:01:05.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
23502026-03-24T21:01:05.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
23512026-03-24T21:01:05.709ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
23522026-03-24T21:01:05.709ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23532026-03-24T21:01:05.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
23542026-03-24T21:01:05.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
23552026-03-24T21:01:05.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23562026-03-24T21:01:05.711ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23572026-03-24T21:01:05.715ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
23582026-03-24T21:01:05.715ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
23592026-03-24T21:01:05.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
23602026-03-24T21:01:05.715ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
23612026-03-24T21:01:05.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
23622026-03-24T21:01:05.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
23632026-03-24T21:01:05.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
23642026-03-24T21:01:05.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
23652026-03-24T21:01:05.727ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
23662026-03-24T21:01:05.731ZINFOnexus (BackgroundTasks): Loaded 0 active trust quorum configurations from database background_task = trust_quorum_manager bgtask = TrustQuorumManager file = nexus/src/app/background/tasks/trust_quorum.rs:83
23672026-03-24T21:01:05.731ZDEBGnexus (BackgroundTasks): activation complete background_task = trust_quorum_manager elapsed = 156.006787ms iteration = 5 status = {"PerRackStatus":{"errors":[],"statuses":[]}}
23682026-03-24T21:01:05.735ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f9046a3-f1cd-40ac-ae8f-c12e0ff06156", "content-length": "691", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} })
23692026-03-24T21:01:05.735ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
23702026-03-24T21:01:05.735ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
23712026-03-24T21:01:05.735ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
23722026-03-24T21:01:05.735ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 202.688625ms iteration = 9 status = {}
23732026-03-24T21:01:05.750ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23742026-03-24T21:01:05.750ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
23752026-03-24T21:01:05.750ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 196.515939ms iteration = 9 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
23762026-03-24T21:01:05.757ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = SledAgent step id = Ensure
23772026-03-24T21:01:05.759ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
23782026-03-24T21:01:05.763ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "fbb4ffb8-7fd5-4bac-b404-692820e96b00", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} }) sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
23792026-03-24T21:01:05.764ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = FirewallRules step id = Ensure
23802026-03-24T21:01:05.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23812026-03-24T21:01:05.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23822026-03-24T21:01:05.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23832026-03-24T21:01:05.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23842026-03-24T21:01:05.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23852026-03-24T21:01:05.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23862026-03-24T21:01:05.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23872026-03-24T21:01:05.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23882026-03-24T21:01:05.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23892026-03-24T21:01:05.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23902026-03-24T21:01:05.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23912026-03-24T21:01:05.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23922026-03-24T21:01:05.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23932026-03-24T21:01:05.786ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
23942026-03-24T21:01:05.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
23952026-03-24T21:01:05.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
23962026-03-24T21:01:05.787ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
23972026-03-24T21:01:05.787ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
23982026-03-24T21:01:05.787ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "02957c25-c993-4d95-8c8d-f0fcd430fdfb", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} })
23992026-03-24T21:01:05.787ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24002026-03-24T21:01:05.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24012026-03-24T21:01:05.790ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24022026-03-24T21:01:05.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24032026-03-24T21:01:05.797ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24042026-03-24T21:01:05.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24052026-03-24T21:01:05.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24062026-03-24T21:01:05.801ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
24072026-03-24T21:01:05.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24082026-03-24T21:01:05.801ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
24092026-03-24T21:01:05.801ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24102026-03-24T21:01:05.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24112026-03-24T21:01:05.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24122026-03-24T21:01:05.806ZDEBGnexus (BackgroundTasks): listed attached subnets background_task = attached_subnet_manager n_subnets = 0
24132026-03-24T21:01:05.807ZDEBGnexus (BackgroundTasks): activation complete background_task = attached_subnet_manager elapsed = 231.107952ms iteration = 5 status = {"db_error":null,"dendrite":{"switch0":{"errors":[],"n_subnets_added":0,"n_subnets_removed":0,"n_total_subnets":0}},"sled":{}}
24142026-03-24T21:01:05.808ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
24152026-03-24T21:01:05.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24162026-03-24T21:01:05.809ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5a24c6ec-8ec2-46f0-95de-10aaeb4115f9", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} })
24172026-03-24T21:01:05.809ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 143.931141ms iteration = 14 status = {}
24182026-03-24T21:01:05.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24192026-03-24T21:01:05.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24202026-03-24T21:01:05.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24212026-03-24T21:01:05.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24222026-03-24T21:01:05.818ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
24232026-03-24T21:01:05.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24242026-03-24T21:01:05.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24252026-03-24T21:01:05.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24262026-03-24T21:01:05.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24272026-03-24T21:01:05.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24282026-03-24T21:01:05.828ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
24292026-03-24T21:01:05.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24302026-03-24T21:01:05.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24312026-03-24T21:01:05.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24322026-03-24T21:01:05.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24332026-03-24T21:01:05.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24342026-03-24T21:01:05.837ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
24352026-03-24T21:01:05.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24362026-03-24T21:01:05.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24372026-03-24T21:01:05.837ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
24382026-03-24T21:01:05.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
24392026-03-24T21:01:05.839ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 306.591158ms iteration = 16 status = {"physical_disks_added":0}
24402026-03-24T21:01:05.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24412026-03-24T21:01:05.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24422026-03-24T21:01:05.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24432026-03-24T21:01:05.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24442026-03-24T21:01:05.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24452026-03-24T21:01:05.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24462026-03-24T21:01:05.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24472026-03-24T21:01:05.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24482026-03-24T21:01:05.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24492026-03-24T21:01:05.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
24502026-03-24T21:01:05.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24512026-03-24T21:01:05.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24522026-03-24T21:01:05.868ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "213baf90-7812-4519-a97c-fa2dce9816e9", "date": "Tue, 24 Mar 2026 21:01:05 GMT"} })
24532026-03-24T21:01:05.868ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
24542026-03-24T21:01:05.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24552026-03-24T21:01:05.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24562026-03-24T21:01:05.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24572026-03-24T21:01:05.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24582026-03-24T21:01:05.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24592026-03-24T21:01:05.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24602026-03-24T21:01:05.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24612026-03-24T21:01:05.874ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
24622026-03-24T21:01:05.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24632026-03-24T21:01:05.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24642026-03-24T21:01:05.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24652026-03-24T21:01:05.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24662026-03-24T21:01:05.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24672026-03-24T21:01:05.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24682026-03-24T21:01:05.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24692026-03-24T21:01:05.879ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
24702026-03-24T21:01:05.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24712026-03-24T21:01:05.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24722026-03-24T21:01:05.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24732026-03-24T21:01:05.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24742026-03-24T21:01:05.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24752026-03-24T21:01:05.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24762026-03-24T21:01:05.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24772026-03-24T21:01:05.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24782026-03-24T21:01:05.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24792026-03-24T21:01:05.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24802026-03-24T21:01:05.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24812026-03-24T21:01:05.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24822026-03-24T21:01:05.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
24832026-03-24T21:01:05.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
24842026-03-24T21:01:05.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24852026-03-24T21:01:05.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24862026-03-24T21:01:05.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24872026-03-24T21:01:05.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24882026-03-24T21:01:05.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24892026-03-24T21:01:05.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24902026-03-24T21:01:05.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24912026-03-24T21:01:05.895ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
24922026-03-24T21:01:05.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24932026-03-24T21:01:05.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
24942026-03-24T21:01:05.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24952026-03-24T21:01:05.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24962026-03-24T21:01:05.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24972026-03-24T21:01:05.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
24982026-03-24T21:01:05.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
24992026-03-24T21:01:05.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25002026-03-24T21:01:05.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25012026-03-24T21:01:05.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25022026-03-24T21:01:05.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25032026-03-24T21:01:05.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25042026-03-24T21:01:05.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25052026-03-24T21:01:05.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25062026-03-24T21:01:05.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25072026-03-24T21:01:05.906ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:118 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
25082026-03-24T21:01:05.906ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 352.473085ms iteration = 5 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
25092026-03-24T21:01:05.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25102026-03-24T21:01:05.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25112026-03-24T21:01:05.906ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
25122026-03-24T21:01:05.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25132026-03-24T21:01:05.913ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
25142026-03-24T21:01:05.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25152026-03-24T21:01:05.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25162026-03-24T21:01:05.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25172026-03-24T21:01:05.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25182026-03-24T21:01:05.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25192026-03-24T21:01:05.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25202026-03-24T21:01:05.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25212026-03-24T21:01:05.922ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
25222026-03-24T21:01:05.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25232026-03-24T21:01:05.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25242026-03-24T21:01:05.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25252026-03-24T21:01:05.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25262026-03-24T21:01:05.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25272026-03-24T21:01:05.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25282026-03-24T21:01:05.930ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25292026-03-24T21:01:05.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25302026-03-24T21:01:05.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25312026-03-24T21:01:05.933ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25322026-03-24T21:01:05.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25332026-03-24T21:01:05.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25342026-03-24T21:01:05.936ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25352026-03-24T21:01:05.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25362026-03-24T21:01:05.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25372026-03-24T21:01:05.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25382026-03-24T21:01:05.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25392026-03-24T21:01:05.941ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 409.287806ms iteration = 9 status = {}
25402026-03-24T21:01:05.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25412026-03-24T21:01:05.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25422026-03-24T21:01:05.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25432026-03-24T21:01:05.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25442026-03-24T21:01:05.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25452026-03-24T21:01:05.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25462026-03-24T21:01:05.958ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
25472026-03-24T21:01:05.958ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25482026-03-24T21:01:05.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25492026-03-24T21:01:05.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25502026-03-24T21:01:05.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25512026-03-24T21:01:05.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25522026-03-24T21:01:05.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25532026-03-24T21:01:05.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25542026-03-24T21:01:05.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25552026-03-24T21:01:05.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25562026-03-24T21:01:05.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25572026-03-24T21:01:05.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25582026-03-24T21:01:05.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25592026-03-24T21:01:05.969ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
25602026-03-24T21:01:05.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25612026-03-24T21:01:05.970ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
25622026-03-24T21:01:05.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25632026-03-24T21:01:05.977ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
25642026-03-24T21:01:05.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25652026-03-24T21:01:05.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25662026-03-24T21:01:05.979ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25672026-03-24T21:01:05.981ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
25682026-03-24T21:01:05.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25692026-03-24T21:01:05.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25702026-03-24T21:01:05.983ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
25712026-03-24T21:01:05.983ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
25722026-03-24T21:01:05.983ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
25732026-03-24T21:01:06.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25742026-03-24T21:01:06.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25752026-03-24T21:01:06.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25762026-03-24T21:01:06.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25772026-03-24T21:01:06.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25782026-03-24T21:01:06.034ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
25792026-03-24T21:01:06.034ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25802026-03-24T21:01:06.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25812026-03-24T21:01:06.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25822026-03-24T21:01:06.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25832026-03-24T21:01:06.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25842026-03-24T21:01:06.042ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
25852026-03-24T21:01:06.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25862026-03-24T21:01:06.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25872026-03-24T21:01:06.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25882026-03-24T21:01:06.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25892026-03-24T21:01:06.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25902026-03-24T21:01:06.049ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
25912026-03-24T21:01:06.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25922026-03-24T21:01:06.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25932026-03-24T21:01:06.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25942026-03-24T21:01:06.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25952026-03-24T21:01:06.053ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25962026-03-24T21:01:06.055ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
25972026-03-24T21:01:06.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25982026-03-24T21:01:06.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25992026-03-24T21:01:06.069ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
26002026-03-24T21:01:06.069ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
26012026-03-24T21:01:06.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26022026-03-24T21:01:06.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26032026-03-24T21:01:06.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26042026-03-24T21:01:06.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26052026-03-24T21:01:06.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26062026-03-24T21:01:06.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
26072026-03-24T21:01:06.076ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
26082026-03-24T21:01:06.077ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "c5b14685-de8a-41da-9756-85bf2d18f2ed", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
26092026-03-24T21:01:06.077ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
26102026-03-24T21:01:06.077ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = Dns step id = Ensure
26112026-03-24T21:01:06.077ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
26122026-03-24T21:01:06.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26132026-03-24T21:01:06.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26142026-03-24T21:01:06.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26152026-03-24T21:01:06.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26162026-03-24T21:01:06.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26172026-03-24T21:01:06.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
26182026-03-24T21:01:06.084ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
26192026-03-24T21:01:06.084ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
26202026-03-24T21:01:06.084ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
26212026-03-24T21:01:06.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26222026-03-24T21:01:06.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26232026-03-24T21:01:06.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26242026-03-24T21:01:06.090ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
26252026-03-24T21:01:06.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26262026-03-24T21:01:06.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26272026-03-24T21:01:06.101ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
26282026-03-24T21:01:06.101ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
26292026-03-24T21:01:06.101ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = b8501e1b-0375-4564-8c26-5f8670be8f45 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
26302026-03-24T21:01:06.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26312026-03-24T21:01:06.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26322026-03-24T21:01:06.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26332026-03-24T21:01:06.106ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
26342026-03-24T21:01:06.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26352026-03-24T21:01:06.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
26362026-03-24T21:01:06.109ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
26372026-03-24T21:01:06.109ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
26382026-03-24T21:01:06.109ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 576.922327ms iteration = 14 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
26392026-03-24T21:01:06.139ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
26402026-03-24T21:01:06.139ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
26412026-03-24T21:01:06.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26422026-03-24T21:01:06.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26432026-03-24T21:01:06.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26442026-03-24T21:01:06.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
26452026-03-24T21:01:06.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26462026-03-24T21:01:06.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26472026-03-24T21:01:06.147ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
26482026-03-24T21:01:06.147ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
26492026-03-24T21:01:06.147ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
26502026-03-24T21:01:06.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26512026-03-24T21:01:06.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26522026-03-24T21:01:06.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26532026-03-24T21:01:06.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26542026-03-24T21:01:06.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26552026-03-24T21:01:06.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
26562026-03-24T21:01:06.154ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
26572026-03-24T21:01:06.154ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
26582026-03-24T21:01:06.154ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
26592026-03-24T21:01:06.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26602026-03-24T21:01:06.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26612026-03-24T21:01:06.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26622026-03-24T21:01:06.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
26632026-03-24T21:01:06.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26642026-03-24T21:01:06.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26652026-03-24T21:01:06.161ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
26662026-03-24T21:01:06.161ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
26672026-03-24T21:01:06.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26682026-03-24T21:01:06.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26692026-03-24T21:01:06.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26702026-03-24T21:01:06.165ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
26712026-03-24T21:01:06.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26722026-03-24T21:01:06.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26732026-03-24T21:01:06.166ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
26742026-03-24T21:01:06.166ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
26752026-03-24T21:01:06.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26762026-03-24T21:01:06.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26772026-03-24T21:01:06.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26782026-03-24T21:01:06.170ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
26792026-03-24T21:01:06.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26802026-03-24T21:01:06.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26812026-03-24T21:01:06.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26822026-03-24T21:01:06.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26832026-03-24T21:01:06.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26842026-03-24T21:01:06.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26852026-03-24T21:01:06.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26862026-03-24T21:01:06.177ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
26872026-03-24T21:01:06.178ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
26882026-03-24T21:01:06.178ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
26892026-03-24T21:01:06.178ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = OmicronZones step id = Cleanup
26902026-03-24T21:01:06.178ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = OmicronZones step id = Cleanup
26912026-03-24T21:01:06.178ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = PhysicalDisks step id = Cleanup
26922026-03-24T21:01:06.179ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = Clickhouse step id = Ensure
26932026-03-24T21:01:06.179ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = Clickhouse step id = Ensure
26942026-03-24T21:01:06.179ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
26952026-03-24T21:01:06.185ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "f2b3760c-573d-4aef-b558-16c18f2914dc", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
26962026-03-24T21:01:06.185ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = SupportBundles step id = Cleanup
26972026-03-24T21:01:06.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26982026-03-24T21:01:06.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26992026-03-24T21:01:06.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27002026-03-24T21:01:06.188ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
27012026-03-24T21:01:06.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
27022026-03-24T21:01:06.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
27032026-03-24T21:01:06.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
27042026-03-24T21:01:06.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
27052026-03-24T21:01:06.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27062026-03-24T21:01:06.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
27072026-03-24T21:01:06.249ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = OmicronZones step id = Cleanup
27082026-03-24T21:01:06.249ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
27092026-03-24T21:01:06.249ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
27102026-03-24T21:01:06.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
27112026-03-24T21:01:06.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
27122026-03-24T21:01:06.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27132026-03-24T21:01:06.252ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
27142026-03-24T21:01:06.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
27152026-03-24T21:01:06.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
27162026-03-24T21:01:06.296ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
27172026-03-24T21:01:06.296ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9e1c09c1-3960-4ecf-a904-19e137699ab3 (blueprint), false)
27182026-03-24T21:01:06.296ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = Cockroach step id = Ensure
27192026-03-24T21:01:06.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
27202026-03-24T21:01:06.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
27212026-03-24T21:01:06.328ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
27222026-03-24T21:01:06.328ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b8f8dd71-595f-4dd4-8e58-4c5400f76bf2 step component = MgsUpdates step id = Ensure
27232026-03-24T21:01:06.328ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 7 reason = Dependency
27242026-03-24T21:01:06.328ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 14 reason = Dependency
27252026-03-24T21:01:06.328ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
27262026-03-24T21:01:06.328ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27272026-03-24T21:01:06.328ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 807.443142ms iteration = 6 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3630,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":96083971,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":96083921,"secs":0}},"event_index":1,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":96107152,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":20794631,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":20794601,"secs":0}},"event_index":2,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":116921323,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":119884833,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":119884793,"secs":0}},"event_index":3,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":236817106,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6207987,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":6207947,"secs":0}},"event_index":4,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":243050164,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":313493204,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":313493164,"secs":0}},"event_index":5,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":556553297,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":101410964,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":101410924,"secs":0}},"event_index":6,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":657974762,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5410,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5370,"secs":0}},"event_index":7,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":657990113,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2340,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2300,"secs":0}},"event_index":8,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":657994193,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3180,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3140,"secs":0}},"event_index":9,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":657998703,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1510,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1480,"secs":0}},"event_index":10,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":658003333,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6410833,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":6410793,"secs":0}},"event_index":11,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":664415156,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":64388356,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":64388316,"secs":0}},"event_index":12,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":728810773,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":46901975,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":46901915,"secs":0}},"event_index":13,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":775731388,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":31394176,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":31394147,"secs":0}},"event_index":14,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":807134544,"secs":0}},{"data":{"attempt_elapsed":{"nanos":4920,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4880,"secs":0}},"event_index":15,"execution_id":"fcd677ad-8417-4497-8a2f-b5ab73ece1d0","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":807146795,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3"}
27282026-03-24T21:01:06.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27292026-03-24T21:01:06.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
27302026-03-24T21:01:06.330ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27312026-03-24T21:01:06.330ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27322026-03-24T21:01:06.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
27332026-03-24T21:01:06.332ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27342026-03-24T21:01:06.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27352026-03-24T21:01:06.332ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
27362026-03-24T21:01:06.332ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
27372026-03-24T21:01:06.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
27382026-03-24T21:01:06.335ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(169e21c7-693b-458e-b4c2-84652769bc8b, 0), (9e2d3a04-ad9f-482d-920e-750ede9ddc94, 0), (80ec73ed-5e31-4e68-b930-28453af567b6, 0), (91efef4f-633d-4073-a5f1-2a6928f85757, 0)] collection_id = 169e21c7-693b-458e-b4c2-84652769bc8b
27392026-03-24T21:01:06.335ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
27402026-03-24T21:01:06.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
27412026-03-24T21:01:06.382ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27422026-03-24T21:01:06.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27432026-03-24T21:01:06.384ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27442026-03-24T21:01:06.386ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
27452026-03-24T21:01:06.386ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27462026-03-24T21:01:06.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27472026-03-24T21:01:06.436ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
27482026-03-24T21:01:06.436ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 108.286559ms iteration = 14 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
27492026-03-24T21:01:06.487ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 169e21c7-693b-458e-b4c2-84652769bc8b file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
27502026-03-24T21:01:06.487ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
27512026-03-24T21:01:06.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
27522026-03-24T21:01:06.492ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(9e2d3a04-ad9f-482d-920e-750ede9ddc94, 0), (80ec73ed-5e31-4e68-b930-28453af567b6, 0), (91efef4f-633d-4073-a5f1-2a6928f85757, 0), (d15a0721-5767-43a2-a3d1-3b91a3df6357, 0)] collection_id = 9e2d3a04-ad9f-482d-920e-750ede9ddc94
27532026-03-24T21:01:06.492ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
27542026-03-24T21:01:06.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
27552026-03-24T21:01:06.617ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 9e2d3a04-ad9f-482d-920e-750ede9ddc94 file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
27562026-03-24T21:01:06.617ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
27572026-03-24T21:01:06.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
27582026-03-24T21:01:06.621ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(80ec73ed-5e31-4e68-b930-28453af567b6, 0), (91efef4f-633d-4073-a5f1-2a6928f85757, 0), (d15a0721-5767-43a2-a3d1-3b91a3df6357, 0), (e0be8d7c-c2e1-4aab-8e77-8dbb701a026c, 0)] collection_id = 80ec73ed-5e31-4e68-b930-28453af567b6
27592026-03-24T21:01:06.621ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
27602026-03-24T21:01:06.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
27612026-03-24T21:01:06.734ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 80ec73ed-5e31-4e68-b930-28453af567b6 file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
27622026-03-24T21:01:06.734ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
27632026-03-24T21:01:06.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
27642026-03-24T21:01:06.739ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(91efef4f-633d-4073-a5f1-2a6928f85757, 0), (d15a0721-5767-43a2-a3d1-3b91a3df6357, 0), (e0be8d7c-c2e1-4aab-8e77-8dbb701a026c, 0)]
27652026-03-24T21:01:06.740ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1052, tv_nsec: 954399194 } })
27662026-03-24T21:01:06.741ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1345db7a-abc1-4a4c-a49c-e6c56102e521.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1920db37-87f5-4a96-b3ef-1db2eb59e9a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6b02d2eb-4373-4f12-8efc-4c0e3f4fa77b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("709c4704-f768-4640-a198-a23621ed6e11.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1052, tv_nsec: 955974506 } })
27672026-03-24T21:01:06.742ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
27682026-03-24T21:01:06.742ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
27692026-03-24T21:01:06.742ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
27702026-03-24T21:01:06.742ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b5a45cb-5150-40d1-a02f-fe8e0cf2b7ac", "content-length": "476", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27712026-03-24T21:01:06.742ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
27722026-03-24T21:01:06.742ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cce9fb4-6839-4326-b407-7de6f40ce8e6", "content-length": "734", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27732026-03-24T21:01:06.743ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
27742026-03-24T21:01:06.743ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "39d89df6-f70e-41b2-af8d-c88d5a88c734", "content-length": "111", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27752026-03-24T21:01:06.743ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
27762026-03-24T21:01:06.744ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "787a6233-bef4-4543-9074-e4a17570088f", "content-length": "111", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27772026-03-24T21:01:06.744ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
27782026-03-24T21:01:06.745ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "699b1b36-6eee-49f6-8043-8a9a533d8f58", "content-length": "163", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27792026-03-24T21:01:06.745ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
27802026-03-24T21:01:06.745ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a59aa548-1d40-4781-abcd-d98972de0995", "content-length": "163", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27812026-03-24T21:01:06.745ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
27822026-03-24T21:01:06.745ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "05b82da1-6842-4d67-9786-08bed0a2864b", "content-length": "166", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27832026-03-24T21:01:06.745ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
27842026-03-24T21:01:06.746ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0624065d-b117-44c7-8435-7d791b10081b", "content-length": "166", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27852026-03-24T21:01:06.746ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
27862026-03-24T21:01:06.746ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "39274bc7-38b4-4079-a32b-d3c4b403b1d0", "content-length": "702", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27872026-03-24T21:01:06.746ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
27882026-03-24T21:01:06.746ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a6f0a8a-bcab-4316-92a7-b6c3745fa17c", "content-length": "727", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27892026-03-24T21:01:06.746ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
27902026-03-24T21:01:06.747ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "26c0498a-bf43-4045-96fe-9e5c7f8743fc", "content-length": "729", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27912026-03-24T21:01:06.747ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
27922026-03-24T21:01:06.747ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e8be8c4-56b6-45cd-bdd4-3ed8f27be1d3", "content-length": "728", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27932026-03-24T21:01:06.747ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
27942026-03-24T21:01:06.747ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fda9a0c-8120-433a-8cc5-ce542b68e5b5", "content-length": "722", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27952026-03-24T21:01:06.747ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
27962026-03-24T21:01:06.747ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6435d66-d58b-48f6-90fb-c02918495580", "content-length": "10", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27972026-03-24T21:01:06.747ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
27982026-03-24T21:01:06.747ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5120bba-cc61-4a9b-a4c4-4c4622e27d86", "content-length": "144", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
27992026-03-24T21:01:06.747ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
28002026-03-24T21:01:06.748ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8f0913b-4d99-487c-9f6e-c092f2d524cd", "content-length": "147", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28012026-03-24T21:01:06.748ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
28022026-03-24T21:01:06.748ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ed2bf42-322a-41b3-82bb-c51d6a208ecb", "content-length": "109", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28032026-03-24T21:01:06.748ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
28042026-03-24T21:01:06.749ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "37fd1105-e3ba-40af-a509-57292ff01040", "content-length": "109", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28052026-03-24T21:01:06.749ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
28062026-03-24T21:01:06.749ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f23a0f0-c12b-4396-bdad-32d0173c4312", "content-length": "163", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28072026-03-24T21:01:06.749ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
28082026-03-24T21:01:06.750ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "663aa9d5-db0a-4e82-aa34-d35b83466702", "content-length": "163", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28092026-03-24T21:01:06.750ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
28102026-03-24T21:01:06.751ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "52b8c36a-989e-431a-8b14-45ac32a4f6cb", "content-length": "166", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28112026-03-24T21:01:06.751ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
28122026-03-24T21:01:06.751ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9404d534-d2de-485d-9e3d-f93f017a882b", "content-length": "166", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28132026-03-24T21:01:06.751ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
28142026-03-24T21:01:06.751ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bed80ad-63ea-471b-9b6b-a260b8285b30", "content-length": "702", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28152026-03-24T21:01:06.751ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
28162026-03-24T21:01:06.752ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "14871bbb-c888-4d91-b6d2-4cdc2d3e3dce", "content-length": "727", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28172026-03-24T21:01:06.752ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
28182026-03-24T21:01:06.752ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0bfb7e7-4f36-4565-9346-6f2c3c63439f", "content-length": "729", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28192026-03-24T21:01:06.752ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
28202026-03-24T21:01:06.752ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "73b20435-7636-4396-bfe4-15fe0412b143", "content-length": "728", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28212026-03-24T21:01:06.752ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
28222026-03-24T21:01:06.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
28232026-03-24T21:01:06.754ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28242026-03-24T21:01:06.756ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
28252026-03-24T21:01:06.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
28262026-03-24T21:01:06.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
28272026-03-24T21:01:06.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28282026-03-24T21:01:06.760ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
28292026-03-24T21:01:06.760ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
28302026-03-24T21:01:06.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
28312026-03-24T21:01:06.763ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
28322026-03-24T21:01:06.763ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
28332026-03-24T21:01:06.826ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5f2c1a6-663e-48cd-ab08-57f41ba751ec", "content-length": "110670", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28342026-03-24T21:01:06.827ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
28352026-03-24T21:01:06.827ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
28362026-03-24T21:01:06.827ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
28372026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
28382026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
28392026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
28402026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
28412026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
28422026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "406a1d7d-cd9a-48ef-b7d9-6cabcc669c47", "content-length": "64", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28432026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
28442026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5475b7dd-b560-4855-8786-95185e7e50ff", "content-length": "64", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28452026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
28462026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "70493053-90d8-4441-b96e-2717286ce99a", "content-length": "64", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28472026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
28482026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "59b80ad6-02cf-4c64-bf5e-daad55471e8d", "content-length": "64", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28492026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
28502026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac6a54a0-e115-4fb1-a46e-7557d762daa0", "content-length": "64", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28512026-03-24T21:01:06.827ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
28522026-03-24T21:01:06.833ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "28305034-b6e6-4da8-808f-ba8bc8ea14c6", "content-length": "384767", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28532026-03-24T21:01:06.835ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "232e22e3-138d-4325-978c-587a3787be17", "content-length": "421195", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28542026-03-24T21:01:06.836ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1afed7e-0faa-4e86-910c-fcb08d1112fd", "content-length": "395085", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28552026-03-24T21:01:06.836ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa7d4ef5-0ee3-4cd2-8a71-3661cfce27b4", "content-length": "389895", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28562026-03-24T21:01:06.837ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "9da7a6e3-c25b-4070-856c-94abc3e0eb23", "content-length": "389447", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28572026-03-24T21:01:06.840ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
28582026-03-24T21:01:06.840ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
28592026-03-24T21:01:06.840ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
28602026-03-24T21:01:06.840ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
28612026-03-24T21:01:06.840ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
28622026-03-24T21:01:06.841ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 (omicron_zone)
28632026-03-24T21:01:06.841ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
28642026-03-24T21:01:06.844ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "f71dccae-f1e7-4cb0-85c5-9d2738d26452", "content-length": "123", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28652026-03-24T21:01:06.844ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
28662026-03-24T21:01:06.846ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
28672026-03-24T21:01:06.846ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
28682026-03-24T21:01:06.847ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a439b94d-d338-414c-acc3-795d2e68da02", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28692026-03-24T21:01:06.847ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
28702026-03-24T21:01:06.847ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
28712026-03-24T21:01:06.847ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
28722026-03-24T21:01:06.847ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a1cc7b1-ff8b-4ed1-90f6-c113d6670c2a", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28732026-03-24T21:01:06.847ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
28742026-03-24T21:01:06.847ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
28752026-03-24T21:01:06.847ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
28762026-03-24T21:01:06.848ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3fb1954-4cde-43f7-a805-06209a5bc2fc", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:01:06 GMT"} })
28772026-03-24T21:01:06.848ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
28782026-03-24T21:01:06.848ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
28792026-03-24T21:01:06.848ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
28802026-03-24T21:01:06.848ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
28812026-03-24T21:01:06.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
28822026-03-24T21:01:06.850ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28832026-03-24T21:01:06.853ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
28842026-03-24T21:01:06.853ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
28852026-03-24T21:01:06.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
28862026-03-24T21:01:07.069ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 1b4f62ac-a6d9-4eb1-9a06-e4cde54a1439 file = nexus/db-queries/src/db/datastore/inventory.rs:1757
28872026-03-24T21:01:07.069ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 1b4f62ac-a6d9-4eb1-9a06-e4cde54a1439 time_started = 2026-03-24 21:01:06.742227 UTC
28882026-03-24T21:01:07.069ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 24 reason = Dependency
28892026-03-24T21:01:07.069ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
28902026-03-24T21:01:07.069ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 741.190495ms iteration = 7 status = {"collection_id":"1b4f62ac-a6d9-4eb1-9a06-e4cde54a1439","time_done":"2026-03-24 21:01:06.848474 UTC","time_started":"2026-03-24 21:01:06.742227 UTC"}
28912026-03-24T21:01:07.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
28922026-03-24T21:01:07.071ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28932026-03-24T21:01:07.074ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
28942026-03-24T21:01:07.074ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
28952026-03-24T21:01:07.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
28962026-03-24T21:01:07.143ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
28972026-03-24T21:01:07.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
28982026-03-24T21:01:07.188ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 13 reason = Dependency
28992026-03-24T21:01:07.188ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 119.483542ms iteration = 24 status = {"Loaded":{"collection_id":"29a1b98a-4509-4d5f-a750-afad7bceeaaa","time_loaded":"2026-03-24T21:01:07.069370821Z","time_started":"2026-03-24T21:01:06.792254Z"}}
29002026-03-24T21:01:07.188ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 9 reason = Dependency
29012026-03-24T21:01:07.188ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
29022026-03-24T21:01:07.188ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Dependency
29032026-03-24T21:01:07.188ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 15 reason = Dependency
29042026-03-24T21:01:07.188ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 19.84\u{b5}s iteration = 13 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
29052026-03-24T21:01:07.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29062026-03-24T21:01:07.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29072026-03-24T21:01:07.188ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29082026-03-24T21:01:07.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29092026-03-24T21:01:07.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29102026-03-24T21:01:07.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29112026-03-24T21:01:07.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29122026-03-24T21:01:07.190ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29132026-03-24T21:01:07.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29142026-03-24T21:01:07.192ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
29152026-03-24T21:01:07.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29162026-03-24T21:01:07.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29172026-03-24T21:01:07.192ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
29182026-03-24T21:01:07.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29192026-03-24T21:01:07.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
29202026-03-24T21:01:07.193ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29212026-03-24T21:01:07.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29222026-03-24T21:01:07.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29232026-03-24T21:01:07.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29242026-03-24T21:01:07.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29252026-03-24T21:01:07.195ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29262026-03-24T21:01:07.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29272026-03-24T21:01:07.196ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
29282026-03-24T21:01:07.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29292026-03-24T21:01:07.196ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
29302026-03-24T21:01:07.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29312026-03-24T21:01:07.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29322026-03-24T21:01:07.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29332026-03-24T21:01:07.196ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29342026-03-24T21:01:07.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29352026-03-24T21:01:07.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29362026-03-24T21:01:07.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29372026-03-24T21:01:07.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29382026-03-24T21:01:07.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
29392026-03-24T21:01:07.199ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
29402026-03-24T21:01:07.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
29412026-03-24T21:01:07.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29422026-03-24T21:01:07.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29432026-03-24T21:01:07.202ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
29442026-03-24T21:01:07.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29452026-03-24T21:01:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29462026-03-24T21:01:07.203ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
29472026-03-24T21:01:07.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29482026-03-24T21:01:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29492026-03-24T21:01:07.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29502026-03-24T21:01:07.204ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 15.879488ms iteration = 17 status = {"physical_disks_added":0}
29512026-03-24T21:01:07.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29522026-03-24T21:01:07.206ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
29532026-03-24T21:01:07.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29542026-03-24T21:01:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29552026-03-24T21:01:07.206ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
29562026-03-24T21:01:07.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29572026-03-24T21:01:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29582026-03-24T21:01:07.207ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
29592026-03-24T21:01:07.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29602026-03-24T21:01:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29612026-03-24T21:01:07.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29622026-03-24T21:01:07.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29632026-03-24T21:01:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29642026-03-24T21:01:07.215ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
29652026-03-24T21:01:07.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29662026-03-24T21:01:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29672026-03-24T21:01:07.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29682026-03-24T21:01:07.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29692026-03-24T21:01:07.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29702026-03-24T21:01:07.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29712026-03-24T21:01:07.219ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
29722026-03-24T21:01:07.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29732026-03-24T21:01:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29742026-03-24T21:01:07.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
29752026-03-24T21:01:07.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29762026-03-24T21:01:07.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29772026-03-24T21:01:07.222ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
29782026-03-24T21:01:07.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29792026-03-24T21:01:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29802026-03-24T21:01:07.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29812026-03-24T21:01:07.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29822026-03-24T21:01:07.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29832026-03-24T21:01:07.225ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
29842026-03-24T21:01:07.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29852026-03-24T21:01:07.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29862026-03-24T21:01:07.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29872026-03-24T21:01:07.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29882026-03-24T21:01:07.229ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
29892026-03-24T21:01:07.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29902026-03-24T21:01:07.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29912026-03-24T21:01:07.230ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
29922026-03-24T21:01:07.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29932026-03-24T21:01:07.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29942026-03-24T21:01:07.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29952026-03-24T21:01:07.233ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
29962026-03-24T21:01:07.233ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 44.88ms iteration = 9 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"29a1b98a-4509-4d5f-a750-afad7bceeaaa","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
29972026-03-24T21:01:07.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
29982026-03-24T21:01:07.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29992026-03-24T21:01:07.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30002026-03-24T21:01:07.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30012026-03-24T21:01:07.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30022026-03-24T21:01:07.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30032026-03-24T21:01:07.240ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
30042026-03-24T21:01:07.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30052026-03-24T21:01:07.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30062026-03-24T21:01:07.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30072026-03-24T21:01:07.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30082026-03-24T21:01:07.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30092026-03-24T21:01:07.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
30102026-03-24T21:01:07.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30112026-03-24T21:01:07.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30122026-03-24T21:01:07.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30132026-03-24T21:01:07.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30142026-03-24T21:01:07.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30152026-03-24T21:01:07.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
30162026-03-24T21:01:07.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30172026-03-24T21:01:07.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30182026-03-24T21:01:07.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30192026-03-24T21:01:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30202026-03-24T21:01:07.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30212026-03-24T21:01:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30222026-03-24T21:01:07.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30232026-03-24T21:01:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
30242026-03-24T21:01:07.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30252026-03-24T21:01:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30262026-03-24T21:01:07.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30272026-03-24T21:01:07.285ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
30282026-03-24T21:01:07.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30292026-03-24T21:01:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30302026-03-24T21:01:07.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30312026-03-24T21:01:07.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30322026-03-24T21:01:07.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30332026-03-24T21:01:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30342026-03-24T21:01:07.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30352026-03-24T21:01:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
30362026-03-24T21:01:07.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30372026-03-24T21:01:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30382026-03-24T21:01:07.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30392026-03-24T21:01:07.299ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
30402026-03-24T21:01:07.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30412026-03-24T21:01:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30422026-03-24T21:01:07.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30432026-03-24T21:01:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30442026-03-24T21:01:07.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30452026-03-24T21:01:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30462026-03-24T21:01:07.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30472026-03-24T21:01:07.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
30482026-03-24T21:01:07.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30492026-03-24T21:01:07.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30502026-03-24T21:01:07.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30512026-03-24T21:01:07.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30522026-03-24T21:01:07.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30532026-03-24T21:01:07.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
30542026-03-24T21:01:07.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30552026-03-24T21:01:07.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30562026-03-24T21:01:07.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30572026-03-24T21:01:07.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30582026-03-24T21:01:07.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30592026-03-24T21:01:07.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
30602026-03-24T21:01:07.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30612026-03-24T21:01:07.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30622026-03-24T21:01:07.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30632026-03-24T21:01:07.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30642026-03-24T21:01:07.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30652026-03-24T21:01:07.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
30662026-03-24T21:01:07.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30672026-03-24T21:01:07.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30682026-03-24T21:01:07.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30692026-03-24T21:01:07.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30702026-03-24T21:01:07.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30712026-03-24T21:01:07.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
30722026-03-24T21:01:07.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30732026-03-24T21:01:07.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30742026-03-24T21:01:07.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30752026-03-24T21:01:07.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30762026-03-24T21:01:07.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30772026-03-24T21:01:07.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
30782026-03-24T21:01:07.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30792026-03-24T21:01:07.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30802026-03-24T21:01:07.389ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
30812026-03-24T21:01:07.389ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
30822026-03-24T21:01:07.389ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 9913a615-333d-4556-ae1d-968f7ce4d381 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
30832026-03-24T21:01:07.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30842026-03-24T21:01:07.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30852026-03-24T21:01:07.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30862026-03-24T21:01:07.394ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
30872026-03-24T21:01:07.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30882026-03-24T21:01:07.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30892026-03-24T21:01:07.397ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
30902026-03-24T21:01:07.397ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
30912026-03-24T21:01:07.397ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 208.812369ms iteration = 15 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
30922026-03-24T21:01:08.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 363 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = faa48cc7-c476-49f8-9565-b3116be60e32 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
30932026-03-24T21:01:10.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 50 reason = Timeout
30942026-03-24T21:01:10.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
30952026-03-24T21:01:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
30962026-03-24T21:01:10.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30972026-03-24T21:01:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
30982026-03-24T21:01:10.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
30992026-03-24T21:01:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31002026-03-24T21:01:10.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
31012026-03-24T21:01:10.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.647949ms iteration = 50 status = {"config_updated":false}
31022026-03-24T21:01:15.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
31032026-03-24T21:01:15.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 51 reason = Timeout
31042026-03-24T21:01:15.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31052026-03-24T21:01:15.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31062026-03-24T21:01:15.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31072026-03-24T21:01:15.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
31082026-03-24T21:01:15.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31092026-03-24T21:01:15.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31102026-03-24T21:01:15.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31112026-03-24T21:01:15.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31122026-03-24T21:01:15.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
31132026-03-24T21:01:15.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31142026-03-24T21:01:15.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
31152026-03-24T21:01:15.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31162026-03-24T21:01:15.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31172026-03-24T21:01:15.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
31182026-03-24T21:01:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
31192026-03-24T21:01:15.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.466622ms iteration = 51 status = {"config_updated":false}
31202026-03-24T21:01:15.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31212026-03-24T21:01:15.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
31222026-03-24T21:01:15.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31232026-03-24T21:01:15.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
31242026-03-24T21:01:15.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
31252026-03-24T21:01:15.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31262026-03-24T21:01:15.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31272026-03-24T21:01:15.555ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
31282026-03-24T21:01:15.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
31292026-03-24T21:01:15.557ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.230668ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
31302026-03-24T21:01:15.557ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
31312026-03-24T21:01:15.557ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.074806ms iteration = 26 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
31322026-03-24T21:01:18.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 437 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 2d35a9fa-091f-4960-bc96-e2a1c7bf859d response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
31332026-03-24T21:01:20.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 25 reason = Timeout
31342026-03-24T21:01:20.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
31352026-03-24T21:01:20.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 52 reason = Timeout
31362026-03-24T21:01:20.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31372026-03-24T21:01:20.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
31382026-03-24T21:01:20.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31392026-03-24T21:01:20.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31402026-03-24T21:01:20.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31412026-03-24T21:01:20.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
31422026-03-24T21:01:20.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31432026-03-24T21:01:20.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
31442026-03-24T21:01:20.536ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
31452026-03-24T21:01:20.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31462026-03-24T21:01:20.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
31472026-03-24T21:01:20.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
31482026-03-24T21:01:20.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.105562ms iteration = 52 status = {"config_updated":false}
31492026-03-24T21:01:20.539ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 29a1b98a-4509-4d5f-a750-afad7bceeaaa original_time_started = 2026-03-24 21:01:06.792254 UTC
31502026-03-24T21:01:20.539ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.161053ms iteration = 25 status = {"Loaded":{"collection_id":"29a1b98a-4509-4d5f-a750-afad7bceeaaa","time_loaded":"2026-03-24T21:01:20.532017897Z","time_started":"2026-03-24T21:01:06.792254Z"}}
31512026-03-24T21:01:20.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 18 reason = Timeout
31522026-03-24T21:01:20.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 18 reason = Timeout
31532026-03-24T21:01:20.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
31542026-03-24T21:01:20.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
31552026-03-24T21:01:20.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
31562026-03-24T21:01:20.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
31572026-03-24T21:01:20.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
31582026-03-24T21:01:20.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
31592026-03-24T21:01:20.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31602026-03-24T21:01:20.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31612026-03-24T21:01:20.579ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
31622026-03-24T21:01:20.580ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
31632026-03-24T21:01:20.580ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
31642026-03-24T21:01:20.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
31652026-03-24T21:01:20.581ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.425413ms iteration = 18 status = "NoSitrep"
31662026-03-24T21:01:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
31672026-03-24T21:01:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
31682026-03-24T21:01:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
31692026-03-24T21:01:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
31702026-03-24T21:01:20.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31712026-03-24T21:01:20.585ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
31722026-03-24T21:01:20.588ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
31732026-03-24T21:01:20.588ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
31742026-03-24T21:01:20.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.394821ms iteration = 18 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
31752026-03-24T21:01:23.551ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:42094
31762026-03-24T21:01:23.551ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42094 req_id = 52c43eb3-cdf2-409d-8f06-78f75b94cc5d uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
31772026-03-24T21:01:23.551ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42094 req_id = 52c43eb3-cdf2-409d-8f06-78f75b94cc5d roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
31782026-03-24T21:01:23.551ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42094 req_id = 52c43eb3-cdf2-409d-8f06-78f75b94cc5d resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
31792026-03-24T21:01:23.554ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 3076 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42094 req_id = 52c43eb3-cdf2-409d-8f06-78f75b94cc5d response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
31802026-03-24T21:01:23.554ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42094 req_id = 9b5e91f5-64b2-4f25-844d-d597f3d898cc uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
31812026-03-24T21:01:23.554ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42094 req_id = 9b5e91f5-64b2-4f25-844d-d597f3d898cc roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
31822026-03-24T21:01:23.554ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42094 req_id = 9b5e91f5-64b2-4f25-844d-d597f3d898cc resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
31832026-03-24T21:01:23.557ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2532 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:42094 req_id = 9b5e91f5-64b2-4f25-844d-d597f3d898cc response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
31842026-03-24T21:01:25.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
31852026-03-24T21:01:25.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 53 reason = Timeout
31862026-03-24T21:01:25.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31872026-03-24T21:01:25.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31882026-03-24T21:01:25.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31892026-03-24T21:01:25.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
31902026-03-24T21:01:25.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31912026-03-24T21:01:25.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31922026-03-24T21:01:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
31932026-03-24T21:01:25.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
31942026-03-24T21:01:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
31952026-03-24T21:01:25.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31962026-03-24T21:01:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
31972026-03-24T21:01:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
31982026-03-24T21:01:25.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
31992026-03-24T21:01:25.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
32002026-03-24T21:01:25.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.106599ms iteration = 53 status = {"config_updated":false}
32012026-03-24T21:01:25.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
32022026-03-24T21:01:25.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32032026-03-24T21:01:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
32042026-03-24T21:01:25.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32052026-03-24T21:01:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
32062026-03-24T21:01:25.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
32072026-03-24T21:01:25.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32082026-03-24T21:01:25.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32092026-03-24T21:01:25.555ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
32102026-03-24T21:01:25.555ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.535444ms iteration = 27 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
32112026-03-24T21:01:25.556ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
32122026-03-24T21:01:25.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
32132026-03-24T21:01:25.595ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.463898ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
32142026-03-24T21:01:28.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 383 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 6228e463-77c2-4ac2-8eaa-a7b7d37ba448 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
32152026-03-24T21:01:30.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 54 reason = Timeout
32162026-03-24T21:01:30.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
32172026-03-24T21:01:30.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
32182026-03-24T21:01:30.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32192026-03-24T21:01:30.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
32202026-03-24T21:01:30.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
32212026-03-24T21:01:30.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
32222026-03-24T21:01:30.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
32232026-03-24T21:01:30.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.65216ms iteration = 54 status = {"config_updated":false}
32242026-03-24T21:01:35.521ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 10 reason = Timeout
32252026-03-24T21:01:35.521ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
32262026-03-24T21:01:35.521ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
32272026-03-24T21:01:35.521ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
32282026-03-24T21:01:35.521ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
32292026-03-24T21:01:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
32302026-03-24T21:01:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
32312026-03-24T21:01:35.526ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1081, tv_nsec: 740896053 } })
32322026-03-24T21:01:35.527ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.31993ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
32332026-03-24T21:01:35.529ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
32342026-03-24T21:01:35.529ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 8.012326ms iteration = 10 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
32352026-03-24T21:01:35.531ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1081, tv_nsec: 746106383 } })
32362026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
32372026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
32382026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
32392026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32402026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32412026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
32422026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 15 reason = Timeout
32432026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32442026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
32452026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
32462026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32472026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32482026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 55 reason = Timeout
32492026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
32502026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 26 reason = Timeout
32512026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
32522026-03-24T21:01:35.532ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
32532026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
32542026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32552026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32562026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32572026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
32582026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
32592026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
32602026-03-24T21:01:35.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32612026-03-24T21:01:35.532ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1081, tv_nsec: 746945466 } })
32622026-03-24T21:01:35.534ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
32632026-03-24T21:01:35.534ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
32642026-03-24T21:01:35.534ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32652026-03-24T21:01:35.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32662026-03-24T21:01:35.535ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "55039125-3630-4dfa-9265-9a6a27f9ca6a", "content-length": "26", "date": "Tue, 24 Mar 2026 21:01:35 GMT"} })
32672026-03-24T21:01:35.535ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
32682026-03-24T21:01:35.535ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
32692026-03-24T21:01:35.535ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32702026-03-24T21:01:35.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32712026-03-24T21:01:35.535ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
32722026-03-24T21:01:35.535ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1081, tv_nsec: 749670439 } })
32732026-03-24T21:01:35.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32742026-03-24T21:01:35.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
32752026-03-24T21:01:35.535ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32762026-03-24T21:01:35.537ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 16.724851ms iteration = 10 status = {}
32772026-03-24T21:01:35.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32782026-03-24T21:01:35.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
32792026-03-24T21:01:35.539ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32802026-03-24T21:01:35.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
32812026-03-24T21:01:35.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
32822026-03-24T21:01:35.539ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32832026-03-24T21:01:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32842026-03-24T21:01:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
32852026-03-24T21:01:35.541ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32862026-03-24T21:01:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
32872026-03-24T21:01:35.541ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
32882026-03-24T21:01:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32892026-03-24T21:01:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
32902026-03-24T21:01:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
32912026-03-24T21:01:35.541ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
32922026-03-24T21:01:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
32932026-03-24T21:01:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
32942026-03-24T21:01:35.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
32952026-03-24T21:01:35.542ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
32962026-03-24T21:01:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
32972026-03-24T21:01:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
32982026-03-24T21:01:35.542ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
32992026-03-24T21:01:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33002026-03-24T21:01:35.543ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 29a1b98a-4509-4d5f-a750-afad7bceeaaa original_time_started = 2026-03-24 21:01:06.792254 UTC
33012026-03-24T21:01:35.543ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 11.15794ms iteration = 26 status = {"Loaded":{"collection_id":"29a1b98a-4509-4d5f-a750-afad7bceeaaa","time_loaded":"2026-03-24T21:01:35.532139888Z","time_started":"2026-03-24T21:01:06.792254Z"}}
33022026-03-24T21:01:35.544ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
33032026-03-24T21:01:35.544ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 12.064365ms iteration = 55 status = {"config_updated":false}
33042026-03-24T21:01:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
33052026-03-24T21:01:35.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
33062026-03-24T21:01:35.544ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33072026-03-24T21:01:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
33082026-03-24T21:01:35.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33092026-03-24T21:01:35.544ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33102026-03-24T21:01:35.545ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1081, tv_nsec: 759179655 } })
33112026-03-24T21:01:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
33122026-03-24T21:01:35.545ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
33132026-03-24T21:01:35.545ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1081, tv_nsec: 760038029 } })
33142026-03-24T21:01:35.546ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 14.09909ms iteration = 10 status = 0
33152026-03-24T21:01:35.546ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
33162026-03-24T21:01:35.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
33172026-03-24T21:01:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
33182026-03-24T21:01:35.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33192026-03-24T21:01:35.547ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
33202026-03-24T21:01:35.547ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
33212026-03-24T21:01:35.548ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f46cb1c5-bd74-45ae-9f2b-8de662348ae8", "content-length": "26", "date": "Tue, 24 Mar 2026 21:01:35 GMT"} })
33222026-03-24T21:01:35.548ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
33232026-03-24T21:01:35.548ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
33242026-03-24T21:01:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
33252026-03-24T21:01:35.548ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
33262026-03-24T21:01:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
33272026-03-24T21:01:35.548ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33282026-03-24T21:01:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
33292026-03-24T21:01:35.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33302026-03-24T21:01:35.549ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1081, tv_nsec: 763618895 } })
33312026-03-24T21:01:35.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
33322026-03-24T21:01:35.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
33332026-03-24T21:01:35.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
33342026-03-24T21:01:35.551ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1081, tv_nsec: 765852526 } })
33352026-03-24T21:01:35.553ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33362026-03-24T21:01:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33372026-03-24T21:01:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 10 reason = Timeout
33382026-03-24T21:01:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
33392026-03-24T21:01:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
33402026-03-24T21:01:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
33412026-03-24T21:01:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
33422026-03-24T21:01:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
33432026-03-24T21:01:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
33442026-03-24T21:01:35.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
33452026-03-24T21:01:35.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
33462026-03-24T21:01:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
33472026-03-24T21:01:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
33482026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
33492026-03-24T21:01:35.555ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
33502026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
33512026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
33522026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33532026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
33542026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
33552026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
33562026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
33572026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
33582026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33592026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
33602026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
33612026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
33622026-03-24T21:01:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
33632026-03-24T21:01:35.557ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 10 reason = Timeout
33642026-03-24T21:01:35.557ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
33652026-03-24T21:01:35.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
33662026-03-24T21:01:35.558ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
33672026-03-24T21:01:35.558ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33682026-03-24T21:01:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33692026-03-24T21:01:35.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33702026-03-24T21:01:35.564ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
33712026-03-24T21:01:35.564ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33722026-03-24T21:01:35.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33732026-03-24T21:01:35.567ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
33742026-03-24T21:01:35.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
33752026-03-24T21:01:35.575ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 10 reason = Timeout
33762026-03-24T21:01:35.575ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 24.021\u{b5}s iteration = 10 status = {"disabled":true,"errors":[],"sps":[]}
33772026-03-24T21:01:35.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 19 reason = Timeout
33782026-03-24T21:01:35.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
33792026-03-24T21:01:35.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 19 reason = Timeout
33802026-03-24T21:01:35.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
33812026-03-24T21:01:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
33822026-03-24T21:01:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
33832026-03-24T21:01:35.581ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
33842026-03-24T21:01:35.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
33852026-03-24T21:01:35.585ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33862026-03-24T21:01:35.586ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
33872026-03-24T21:01:35.586ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
33882026-03-24T21:01:35.586ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
33892026-03-24T21:01:35.586ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
33902026-03-24T21:01:35.586ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
33912026-03-24T21:01:35.586ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
33922026-03-24T21:01:35.587ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
33932026-03-24T21:01:35.587ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33942026-03-24T21:01:35.587ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
33952026-03-24T21:01:35.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33962026-03-24T21:01:35.590ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
33972026-03-24T21:01:35.590ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 58.450516ms iteration = 28 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
33982026-03-24T21:01:35.590ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
33992026-03-24T21:01:35.590ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
34002026-03-24T21:01:35.591ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
34012026-03-24T21:01:35.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
34022026-03-24T21:01:35.592ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
34032026-03-24T21:01:35.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
34042026-03-24T21:01:35.595ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34052026-03-24T21:01:35.597ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
34062026-03-24T21:01:35.605ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
34072026-03-24T21:01:35.605ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
34082026-03-24T21:01:35.605ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 29.964098ms iteration = 19 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
34092026-03-24T21:01:35.646ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
34102026-03-24T21:01:35.646ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34112026-03-24T21:01:35.646ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 114.104527ms iteration = 15 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
34122026-03-24T21:01:35.646ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
34132026-03-24T21:01:35.646ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34142026-03-24T21:01:35.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
34152026-03-24T21:01:35.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34162026-03-24T21:01:35.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34172026-03-24T21:01:35.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34182026-03-24T21:01:35.651ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
34192026-03-24T21:01:35.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
34202026-03-24T21:01:35.651ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34212026-03-24T21:01:35.651ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
34222026-03-24T21:01:35.651ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
34232026-03-24T21:01:35.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
34242026-03-24T21:01:35.653ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
34252026-03-24T21:01:35.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
34262026-03-24T21:01:35.653ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
34272026-03-24T21:01:35.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
34282026-03-24T21:01:35.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
34292026-03-24T21:01:35.654ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 100.367416ms iteration = 10 status = {"errors":[],"finish_invoked_ok":[]}
34302026-03-24T21:01:35.654ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 99.75986ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
34312026-03-24T21:01:35.655ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 98.117465ms iteration = 10 status = {"errors":[],"requests_created_ok":[]}
34322026-03-24T21:01:35.656ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 110.162031ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
34332026-03-24T21:01:35.657ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 102.595806ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
34342026-03-24T21:01:35.659ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
34352026-03-24T21:01:35.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
34362026-03-24T21:01:35.662ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
34372026-03-24T21:01:35.662ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 107.794967ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
34382026-03-24T21:01:35.663ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34392026-03-24T21:01:35.667ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
34402026-03-24T21:01:35.667ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34412026-03-24T21:01:35.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34422026-03-24T21:01:35.669ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34432026-03-24T21:01:35.672ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
34442026-03-24T21:01:35.672ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
34452026-03-24T21:01:35.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
34462026-03-24T21:01:35.676ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 100.399978ms iteration = 19 status = "NoSitrep"
34472026-03-24T21:01:35.713ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
34482026-03-24T21:01:35.713ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
34492026-03-24T21:01:35.713ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 158.490833ms iteration = 10 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
34502026-03-24T21:01:35.714ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 159.749038ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
34512026-03-24T21:01:35.720ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 164.480095ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
34522026-03-24T21:01:35.754ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34532026-03-24T21:01:35.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
34542026-03-24T21:01:35.756ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "93841935-bedf-432a-8849-911627987567", "date": "Tue, 24 Mar 2026 21:01:35 GMT"} })
34552026-03-24T21:01:35.756ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
34562026-03-24T21:01:35.756ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34572026-03-24T21:01:35.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34582026-03-24T21:01:35.780ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 248.391728ms iteration = 18 status = {"physical_disks_added":0}
34592026-03-24T21:01:35.787ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
34602026-03-24T21:01:35.787ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
34612026-03-24T21:01:35.788ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a9be8b4-3784-40cc-b5f6-5a147a7cdb1e", "content-length": "372", "date": "Tue, 24 Mar 2026 21:01:35 GMT"} })
34622026-03-24T21:01:35.788ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34632026-03-24T21:01:35.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34642026-03-24T21:01:35.806ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34652026-03-24T21:01:35.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34662026-03-24T21:01:35.810ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34672026-03-24T21:01:35.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34682026-03-24T21:01:35.812ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34692026-03-24T21:01:35.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34702026-03-24T21:01:35.814ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 282.309952ms iteration = 10 status = {}
34712026-03-24T21:01:35.842ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
34722026-03-24T21:01:35.842ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
34732026-03-24T21:01:35.849ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ee265bbd-cbdc-4c5b-bba7-44122cbc9eff", "date": "Tue, 24 Mar 2026 21:01:35 GMT"} })
34742026-03-24T21:01:35.849ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34752026-03-24T21:01:35.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34762026-03-24T21:01:35.851ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34772026-03-24T21:01:35.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34782026-03-24T21:01:35.854ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
34792026-03-24T21:01:35.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
34802026-03-24T21:01:35.858ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
34812026-03-24T21:01:35.858ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
34822026-03-24T21:01:35.858ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fb280e2b-be83-452a-84e2-493f5bb6e998", "date": "Tue, 24 Mar 2026 21:01:35 GMT"} })
34832026-03-24T21:01:35.858ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 303.634717ms iteration = 15 status = {}
34842026-03-24T21:01:35.897ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34852026-03-24T21:01:35.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
34862026-03-24T21:01:35.899ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34872026-03-24T21:01:35.901ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
34882026-03-24T21:01:35.901ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34892026-03-24T21:01:35.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
34902026-03-24T21:01:35.905ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34912026-03-24T21:01:35.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
34922026-03-24T21:01:35.907ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
34932026-03-24T21:01:35.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
34942026-03-24T21:01:36.006ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
34952026-03-24T21:01:36.007ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "712fe1d1-1afc-4fd3-be3f-0d9cb8b759eb", "content-length": "691", "date": "Tue, 24 Mar 2026 21:01:36 GMT"} })
34962026-03-24T21:01:36.007ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
34972026-03-24T21:01:36.007ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
34982026-03-24T21:01:36.007ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
34992026-03-24T21:01:36.007ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 475.141062ms iteration = 10 status = {}
35002026-03-24T21:01:38.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 373 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 14995415-9140-4927-bc38-133c0b8683fc response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
35012026-03-24T21:01:40.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 56 reason = Timeout
35022026-03-24T21:01:40.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35032026-03-24T21:01:40.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35042026-03-24T21:01:40.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35052026-03-24T21:01:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
35062026-03-24T21:01:40.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35072026-03-24T21:01:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35082026-03-24T21:01:40.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
35092026-03-24T21:01:40.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.022629ms iteration = 56 status = {"config_updated":false}
35102026-03-24T21:01:45.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
35112026-03-24T21:01:45.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35122026-03-24T21:01:45.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 57 reason = Timeout
35132026-03-24T21:01:45.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35142026-03-24T21:01:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
35152026-03-24T21:01:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35162026-03-24T21:01:45.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35172026-03-24T21:01:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
35182026-03-24T21:01:45.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35192026-03-24T21:01:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35202026-03-24T21:01:45.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
35212026-03-24T21:01:45.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.680157ms iteration = 57 status = {"config_updated":false}
35222026-03-24T21:01:45.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
35232026-03-24T21:01:45.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35242026-03-24T21:01:45.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35252026-03-24T21:01:45.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35262026-03-24T21:01:45.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35272026-03-24T21:01:45.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35282026-03-24T21:01:45.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
35292026-03-24T21:01:45.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35302026-03-24T21:01:45.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
35312026-03-24T21:01:45.587ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35322026-03-24T21:01:45.589ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
35332026-03-24T21:01:45.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35342026-03-24T21:01:45.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
35352026-03-24T21:01:45.596ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
35362026-03-24T21:01:45.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
35372026-03-24T21:01:45.636ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 82.545066ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
35382026-03-24T21:01:45.708ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
35392026-03-24T21:01:45.708ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 175.449921ms iteration = 29 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
35402026-03-24T21:01:48.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 678 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 133c4329-2eba-46b7-8327-4c31c45c065a response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
35412026-03-24T21:01:50.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 27 reason = Timeout
35422026-03-24T21:01:50.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 58 reason = Timeout
35432026-03-24T21:01:50.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
35442026-03-24T21:01:50.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35452026-03-24T21:01:50.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35462026-03-24T21:01:50.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
35472026-03-24T21:01:50.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35482026-03-24T21:01:50.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35492026-03-24T21:01:50.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
35502026-03-24T21:01:50.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35512026-03-24T21:01:50.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35522026-03-24T21:01:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
35532026-03-24T21:01:50.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
35542026-03-24T21:01:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
35552026-03-24T21:01:50.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
35562026-03-24T21:01:50.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.237285ms iteration = 58 status = {"config_updated":false}
35572026-03-24T21:01:50.540ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 29a1b98a-4509-4d5f-a750-afad7bceeaaa original_time_started = 2026-03-24 21:01:06.792254 UTC
35582026-03-24T21:01:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.261033ms iteration = 27 status = {"Loaded":{"collection_id":"29a1b98a-4509-4d5f-a750-afad7bceeaaa","time_loaded":"2026-03-24T21:01:50.532289162Z","time_started":"2026-03-24T21:01:06.792254Z"}}
35592026-03-24T21:01:50.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 20 reason = Timeout
35602026-03-24T21:01:50.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 20 reason = Timeout
35612026-03-24T21:01:50.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
35622026-03-24T21:01:50.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
35632026-03-24T21:01:50.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
35642026-03-24T21:01:50.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
35652026-03-24T21:01:50.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
35662026-03-24T21:01:50.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
35672026-03-24T21:01:50.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35682026-03-24T21:01:50.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35692026-03-24T21:01:50.579ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
35702026-03-24T21:01:50.579ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
35712026-03-24T21:01:50.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
35722026-03-24T21:01:50.579ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
35732026-03-24T21:01:50.581ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.034262ms iteration = 20 status = "NoSitrep"
35742026-03-24T21:01:50.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
35752026-03-24T21:01:50.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
35762026-03-24T21:01:50.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
35772026-03-24T21:01:50.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
35782026-03-24T21:01:50.622ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35792026-03-24T21:01:50.624ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
35802026-03-24T21:01:50.628ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
35812026-03-24T21:01:50.628ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
35822026-03-24T21:01:50.628ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 53.076771ms iteration = 20 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
35832026-03-24T21:01:55.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
35842026-03-24T21:01:55.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 59 reason = Timeout
35852026-03-24T21:01:55.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35862026-03-24T21:01:55.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35872026-03-24T21:01:55.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35882026-03-24T21:01:55.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
35892026-03-24T21:01:55.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35902026-03-24T21:01:55.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35912026-03-24T21:01:55.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
35922026-03-24T21:01:55.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
35932026-03-24T21:01:55.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
35942026-03-24T21:01:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
35952026-03-24T21:01:55.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
35962026-03-24T21:01:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
35972026-03-24T21:01:55.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
35982026-03-24T21:01:55.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.260625ms iteration = 59 status = {"config_updated":false}
35992026-03-24T21:01:55.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36002026-03-24T21:01:55.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
36012026-03-24T21:01:55.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36022026-03-24T21:01:55.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
36032026-03-24T21:01:55.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
36042026-03-24T21:01:55.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
36052026-03-24T21:01:55.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
36062026-03-24T21:01:55.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36072026-03-24T21:01:55.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36082026-03-24T21:01:55.556ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
36092026-03-24T21:01:55.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
36102026-03-24T21:01:55.557ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
36112026-03-24T21:01:55.557ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.747127ms iteration = 30 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
36122026-03-24T21:01:55.597ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.152987ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
36132026-03-24T21:01:58.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 370 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 52aa91de-7209-49e2-a6e7-8a88e37fb2ce response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
36142026-03-24T21:02:00.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 60 reason = Timeout
36152026-03-24T21:02:00.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
36162026-03-24T21:02:00.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
36172026-03-24T21:02:00.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36182026-03-24T21:02:00.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
36192026-03-24T21:02:00.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
36202026-03-24T21:02:00.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
36212026-03-24T21:02:00.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
36222026-03-24T21:02:00.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.15071ms iteration = 60 status = {"config_updated":false}
36232026-03-24T21:02:05.515ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
36242026-03-24T21:02:05.515ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
36252026-03-24T21:02:05.515ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
36262026-03-24T21:02:05.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
36272026-03-24T21:02:05.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
36282026-03-24T21:02:05.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
36292026-03-24T21:02:05.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36302026-03-24T21:02:05.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
36312026-03-24T21:02:05.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
36322026-03-24T21:02:05.519ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
36332026-03-24T21:02:05.519ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
36342026-03-24T21:02:05.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36352026-03-24T21:02:05.519ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
36362026-03-24T21:02:05.519ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2026-03-24 20:52:05.519946663 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
36372026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
36382026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
36392026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
36402026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
36412026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 11 reason = Timeout
36422026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
36432026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
36442026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
36452026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
36462026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
36472026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
36482026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36492026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
36502026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
36512026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
36522026-03-24T21:02:05.520ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("93b6e989-1f7d-4884-aec9-ecbc076ed6c4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 735094436 } })
36532026-03-24T21:02:05.520ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("27f3a9c5-67c7-41c5-994d-9db513cda557.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2f7ea2ed-fd45-4cd7-9fb1-f48119541118.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 735087586 } })
36542026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
36552026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
36562026-03-24T21:02:05.520ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
36572026-03-24T21:02:05.520ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = ExternalNetworking step id = Ensure
36582026-03-24T21:02:05.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
36592026-03-24T21:02:05.521ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
36602026-03-24T21:02:05.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
36612026-03-24T21:02:05.521ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
36622026-03-24T21:02:05.521ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.683315ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
36632026-03-24T21:02:05.521ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
36642026-03-24T21:02:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
36652026-03-24T21:02:05.521ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
36662026-03-24T21:02:05.521ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.710006ms iteration = 6 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
36672026-03-24T21:02:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
36682026-03-24T21:02:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
36692026-03-24T21:02:05.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
36702026-03-24T21:02:05.523ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e69dd596-d2d8-404d-8746-600e38d46c22", "date": "Tue, 24 Mar 2026 21:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
36712026-03-24T21:02:05.523ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
36722026-03-24T21:02:05.523ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "7719a668-04cd-4f96-b0ba-8917d2bda110", "date": "Tue, 24 Mar 2026 21:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
36732026-03-24T21:02:05.523ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
36742026-03-24T21:02:05.523ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
36752026-03-24T21:02:05.523ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 3.468744ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
36762026-03-24T21:02:05.523ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 737885751 } })
36772026-03-24T21:02:05.523ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
36782026-03-24T21:02:05.524ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 738618741 } })
36792026-03-24T21:02:05.524ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 738790386 } })
36802026-03-24T21:02:05.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
36812026-03-24T21:02:05.524ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2026-03-24 20:52:05.519946663 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
36822026-03-24T21:02:05.524ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.706507ms iteration = 6 status = {"expiration":"2026-03-24T20:52:05.519946663Z","pruned":[]}
36832026-03-24T21:02:05.525ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.103178ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
36842026-03-24T21:02:05.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "a5ca94d2-ef40-4289-80bf-f444c5374e40", "date": "Tue, 24 Mar 2026 21:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
36852026-03-24T21:02:05.525ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
36862026-03-24T21:02:05.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "95dc0bc5-1dc9-44ea-8fcb-42ecf71270a0", "date": "Tue, 24 Mar 2026 21:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
36872026-03-24T21:02:05.525ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
36882026-03-24T21:02:05.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1153a658-771f-4282-a811-ba9b7d0b3cd3", "date": "Tue, 24 Mar 2026 21:02:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
36892026-03-24T21:02:05.525ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
36902026-03-24T21:02:05.525ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
36912026-03-24T21:02:05.525ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 5.422836ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
36922026-03-24T21:02:05.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
36932026-03-24T21:02:05.526ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
36942026-03-24T21:02:05.526ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
36952026-03-24T21:02:05.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
36962026-03-24T21:02:05.526ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "05175450-5be8-4de3-9f36-f2dd07618255", "content-length": "26", "date": "Tue, 24 Mar 2026 21:02:05 GMT"} })
36972026-03-24T21:02:05.526ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
36982026-03-24T21:02:05.526ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
36992026-03-24T21:02:05.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
37002026-03-24T21:02:05.526ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
37012026-03-24T21:02:05.526ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 6.794183ms iteration = 11 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
37022026-03-24T21:02:05.527ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 741380446 } })
37032026-03-24T21:02:05.529ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 9.096025ms iteration = 11 status = {}
37042026-03-24T21:02:05.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
37052026-03-24T21:02:05.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
37062026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 10 reason = Timeout
37072026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Timeout
37082026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37092026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 16 reason = Timeout
37102026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37112026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37122026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
37132026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37142026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 28 reason = Timeout
37152026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
37162026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 16 reason = Timeout
37172026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
37182026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
37192026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
37202026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
37212026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
37222026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
37232026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37242026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 61 reason = Timeout
37252026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
37262026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
37272026-03-24T21:02:05.532ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
37282026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
37292026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37302026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37312026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37322026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
37332026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
37342026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37352026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
37362026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
37372026-03-24T21:02:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
37382026-03-24T21:02:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
37392026-03-24T21:02:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
37402026-03-24T21:02:05.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37412026-03-24T21:02:05.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37422026-03-24T21:02:05.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37432026-03-24T21:02:05.536ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37442026-03-24T21:02:05.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
37452026-03-24T21:02:05.540ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
37462026-03-24T21:02:05.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37472026-03-24T21:02:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37482026-03-24T21:02:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
37492026-03-24T21:02:05.541ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
37502026-03-24T21:02:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
37512026-03-24T21:02:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37522026-03-24T21:02:05.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37532026-03-24T21:02:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
37542026-03-24T21:02:05.546ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
37552026-03-24T21:02:05.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
37562026-03-24T21:02:05.546ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
37572026-03-24T21:02:05.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
37582026-03-24T21:02:05.547ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
37592026-03-24T21:02:05.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37602026-03-24T21:02:05.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37612026-03-24T21:02:05.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
37622026-03-24T21:02:05.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
37632026-03-24T21:02:05.548ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37642026-03-24T21:02:05.549ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
37652026-03-24T21:02:05.551ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
37662026-03-24T21:02:05.552ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
37672026-03-24T21:02:05.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37682026-03-24T21:02:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37692026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
37702026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
37712026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = tuf_repo_pruner iteration = 3 reason = Timeout
37722026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
37732026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
37742026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37752026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
37762026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
37772026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 6 reason = Timeout
37782026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
37792026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
37802026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 2 reason = Timeout
37812026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
37822026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 11 reason = Timeout
37832026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
37842026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
37852026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
37862026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
37872026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
37882026-03-24T21:02:05.554ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
37892026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
37902026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
37912026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
37922026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
37932026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
37942026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
37952026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
37962026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
37972026-03-24T21:02:05.554ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
37982026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
37992026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
38002026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38012026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
38022026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
38032026-03-24T21:02:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38042026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
38052026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
38062026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
38072026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
38082026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
38092026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
38102026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
38112026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
38122026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
38132026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
38142026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38152026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
38162026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38172026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
38182026-03-24T21:02:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
38192026-03-24T21:02:05.555ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
38202026-03-24T21:02:05.556ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38212026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38222026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 14 reason = Timeout
38232026-03-24T21:02:05.557ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
38242026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 6 reason = Timeout
38252026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 6 reason = Timeout
38262026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
38272026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 18.86\u{b5}s iteration = 14 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
38282026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
38292026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 11 reason = Timeout
38302026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
38312026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
38322026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
38332026-03-24T21:02:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
38342026-03-24T21:02:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
38352026-03-24T21:02:05.561ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
38362026-03-24T21:02:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
38372026-03-24T21:02:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
38382026-03-24T21:02:05.562ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
38392026-03-24T21:02:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
38402026-03-24T21:02:05.564ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
38412026-03-24T21:02:05.564ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 32.23896ms iteration = 61 status = {"config_updated":false}
38422026-03-24T21:02:05.564ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
38432026-03-24T21:02:05.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38442026-03-24T21:02:05.568ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
38452026-03-24T21:02:05.570ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
38462026-03-24T21:02:05.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
38472026-03-24T21:02:05.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
38482026-03-24T21:02:05.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
38492026-03-24T21:02:05.571ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38502026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
38512026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
38522026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_rendezvous iteration = 2 reason = Timeout
38532026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_rendezvous elapsed = 13.14\u{b5}s iteration = 2 status = "NoSitrep"
38542026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): activating background_task = session_cleanup iteration = 2 reason = Timeout
38552026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): roles background_task = session_cleanup roles = RoleSet { roles: {} }
38562026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 11 reason = Timeout
38572026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 19.08\u{b5}s iteration = 11 status = {"disabled":true,"errors":[],"sps":[]}
38582026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): activating background_task = trust_quorum_manager iteration = 6 reason = Timeout
38592026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): activating background_task = attached_subnet_manager iteration = 6 reason = Timeout
38602026-03-24T21:02:05.575ZINFOnexus (BackgroundTasks): starting attached subnet manager work background_task = attached_subnet_manager file = nexus/src/app/background/tasks/attached_subnets.rs:374
38612026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 6 reason = Timeout
38622026-03-24T21:02:05.575ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
38632026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
38642026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
38652026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 21 reason = Timeout
38662026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
38672026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 21 reason = Timeout
38682026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
38692026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = session_cleanup resource = Database result = Ok(())
38702026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
38712026-03-24T21:02:05.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
38722026-03-24T21:02:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
38732026-03-24T21:02:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
38742026-03-24T21:02:05.576ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 790454679 } })
38752026-03-24T21:02:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
38762026-03-24T21:02:05.578ZDEBGnexus (BackgroundTasks): roles background_task = attached_subnet_manager roles = RoleSet { roles: {} }
38772026-03-24T21:02:05.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = attached_subnet_manager resource = Database result = Ok(())
38782026-03-24T21:02:05.583ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
38792026-03-24T21:02:05.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
38802026-03-24T21:02:05.590ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
38812026-03-24T21:02:05.590ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38822026-03-24T21:02:05.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38832026-03-24T21:02:05.592ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
38842026-03-24T21:02:05.595ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
38852026-03-24T21:02:05.601ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
38862026-03-24T21:02:05.613ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
38872026-03-24T21:02:05.614ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = SledList step id = Fetch
38882026-03-24T21:02:05.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
38892026-03-24T21:02:05.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
38902026-03-24T21:02:05.635ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38912026-03-24T21:02:05.641ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
38922026-03-24T21:02:05.644ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38932026-03-24T21:02:05.645ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38942026-03-24T21:02:05.647ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38952026-03-24T21:02:05.648ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
38962026-03-24T21:02:05.648ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
38972026-03-24T21:02:05.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
38982026-03-24T21:02:05.649ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 95.29565ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
38992026-03-24T21:02:05.649ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
39002026-03-24T21:02:05.649ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39012026-03-24T21:02:05.650ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39022026-03-24T21:02:05.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
39032026-03-24T21:02:05.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
39042026-03-24T21:02:05.652ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
39052026-03-24T21:02:05.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
39062026-03-24T21:02:05.654ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
39072026-03-24T21:02:05.654ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39082026-03-24T21:02:05.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39092026-03-24T21:02:05.663ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39102026-03-24T21:02:05.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39112026-03-24T21:02:05.664ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
39122026-03-24T21:02:05.664ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
39132026-03-24T21:02:05.664ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
39142026-03-24T21:02:05.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39152026-03-24T21:02:05.665ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 108.065544ms iteration = 11 status = {"errors":[],"requests_created_ok":[]}
39162026-03-24T21:02:05.665ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
39172026-03-24T21:02:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39182026-03-24T21:02:05.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
39192026-03-24T21:02:05.667ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
39202026-03-24T21:02:05.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39212026-03-24T21:02:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
39222026-03-24T21:02:05.668ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 135.906025ms iteration = 11 status = 0
39232026-03-24T21:02:05.668ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
39242026-03-24T21:02:05.668ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 111.203399ms iteration = 6 status = {"by_rx":{},"error":null}
39252026-03-24T21:02:05.668ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
39262026-03-24T21:02:05.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
39272026-03-24T21:02:05.668ZDEBGnexus (BackgroundTasks): roles background_task = session_cleanup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39282026-03-24T21:02:05.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
39292026-03-24T21:02:05.669ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
39302026-03-24T21:02:05.669ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
39312026-03-24T21:02:05.670ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39322026-03-24T21:02:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
39332026-03-24T21:02:05.670ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
39342026-03-24T21:02:05.670ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39352026-03-24T21:02:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
39362026-03-24T21:02:05.671ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39372026-03-24T21:02:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = session_cleanup resource = Fleet result = Ok(())
39382026-03-24T21:02:05.671ZDEBGnexus (BackgroundTasks): roles background_task = session_cleanup roles = RoleSet { roles: {} }
39392026-03-24T21:02:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = session_cleanup resource = Database result = Ok(())
39402026-03-24T21:02:05.672ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
39412026-03-24T21:02:05.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
39422026-03-24T21:02:05.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
39432026-03-24T21:02:05.673ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39442026-03-24T21:02:05.674ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
39452026-03-24T21:02:05.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
39462026-03-24T21:02:05.675ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
39472026-03-24T21:02:05.675ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
39482026-03-24T21:02:05.675ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
39492026-03-24T21:02:05.675ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
39502026-03-24T21:02:05.675ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
39512026-03-24T21:02:05.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
39522026-03-24T21:02:05.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
39532026-03-24T21:02:05.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
39542026-03-24T21:02:05.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
39552026-03-24T21:02:05.677ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
39562026-03-24T21:02:05.677ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39572026-03-24T21:02:05.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39582026-03-24T21:02:05.678ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
39592026-03-24T21:02:05.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
39602026-03-24T21:02:05.678ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
39612026-03-24T21:02:05.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39622026-03-24T21:02:05.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
39632026-03-24T21:02:05.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39642026-03-24T21:02:05.681ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
39652026-03-24T21:02:05.682ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
39662026-03-24T21:02:05.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
39672026-03-24T21:02:05.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
39682026-03-24T21:02:05.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
39692026-03-24T21:02:05.684ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39702026-03-24T21:02:05.684ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
39712026-03-24T21:02:05.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
39722026-03-24T21:02:05.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
39732026-03-24T21:02:05.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39742026-03-24T21:02:05.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
39752026-03-24T21:02:05.687ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39762026-03-24T21:02:05.688ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
39772026-03-24T21:02:05.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39782026-03-24T21:02:05.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39792026-03-24T21:02:05.689ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39802026-03-24T21:02:05.689ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
39812026-03-24T21:02:05.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
39822026-03-24T21:02:05.692ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
39832026-03-24T21:02:05.692ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
39842026-03-24T21:02:05.692ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
39852026-03-24T21:02:05.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
39862026-03-24T21:02:05.693ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
39872026-03-24T21:02:05.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
39882026-03-24T21:02:05.693ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39892026-03-24T21:02:05.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
39902026-03-24T21:02:05.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
39912026-03-24T21:02:05.695ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
39922026-03-24T21:02:05.696ZDEBGnexus (BackgroundTasks): session cleanup found no expired sessions background_task = session_cleanup cutoff = 2026-03-23 21:02:05.575349247 UTC
39932026-03-24T21:02:05.696ZDEBGnexus (BackgroundTasks): activation complete background_task = session_cleanup elapsed = 120.808588ms iteration = 2 status = {"cutoff":"2026-03-23T21:02:05.575349247Z","deleted":0,"error":null,"limit":10000}
39942026-03-24T21:02:05.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
39952026-03-24T21:02:05.696ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
39962026-03-24T21:02:05.699ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
39972026-03-24T21:02:05.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
39982026-03-24T21:02:05.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
39992026-03-24T21:02:05.703ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 29a1b98a-4509-4d5f-a750-afad7bceeaaa original_time_started = 2026-03-24 21:01:06.792254 UTC
40002026-03-24T21:02:05.703ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 171.154325ms iteration = 28 status = {"Loaded":{"collection_id":"29a1b98a-4509-4d5f-a750-afad7bceeaaa","time_loaded":"2026-03-24T21:02:05.532134092Z","time_started":"2026-03-24T21:01:06.792254Z"}}
40012026-03-24T21:02:05.704ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 128.612858ms iteration = 21 status = "NoSitrep"
40022026-03-24T21:02:05.704ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40032026-03-24T21:02:05.705ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
40042026-03-24T21:02:05.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
40052026-03-24T21:02:05.707ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
40062026-03-24T21:02:05.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
40072026-03-24T21:02:05.708ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 922644883 } })
40082026-03-24T21:02:05.708ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
40092026-03-24T21:02:05.708ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
40102026-03-24T21:02:05.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
40112026-03-24T21:02:05.709ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
40122026-03-24T21:02:05.709ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 923860186 } })
40132026-03-24T21:02:05.710ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40142026-03-24T21:02:05.711ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
40152026-03-24T21:02:05.711ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40162026-03-24T21:02:05.712ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40172026-03-24T21:02:05.712ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c13b52b5-a8cd-4cee-8913-2a535cb12ce6", "content-length": "26", "date": "Tue, 24 Mar 2026 21:02:05 GMT"} })
40182026-03-24T21:02:05.712ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40192026-03-24T21:02:05.712ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
40202026-03-24T21:02:05.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40212026-03-24T21:02:05.713ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40222026-03-24T21:02:05.714ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 928739548 } })
40232026-03-24T21:02:05.714ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
40242026-03-24T21:02:05.714ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40252026-03-24T21:02:05.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40262026-03-24T21:02:05.717ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1111, tv_nsec: 931556224 } })
40272026-03-24T21:02:05.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40282026-03-24T21:02:05.719ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40292026-03-24T21:02:05.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40302026-03-24T21:02:05.720ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
40312026-03-24T21:02:05.720ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
40322026-03-24T21:02:05.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
40332026-03-24T21:02:05.722ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
40342026-03-24T21:02:05.722ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
40352026-03-24T21:02:05.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
40362026-03-24T21:02:05.729ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 172.177592ms iteration = 6 status = {"dispatched":[],"errors":[],"glob_version":"242.0.0","globs_reprocessed":{},"no_receivers":[]}
40372026-03-24T21:02:05.730ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40382026-03-24T21:02:05.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40392026-03-24T21:02:05.733ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 179.346516ms iteration = 11 status = {"errors":[],"finish_invoked_ok":[]}
40402026-03-24T21:02:05.734ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40412026-03-24T21:02:05.734ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
40422026-03-24T21:02:05.734ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40432026-03-24T21:02:05.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
40442026-03-24T21:02:05.734ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
40452026-03-24T21:02:05.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
40462026-03-24T21:02:05.736ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
40472026-03-24T21:02:05.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
40482026-03-24T21:02:05.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40492026-03-24T21:02:05.739ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
40502026-03-24T21:02:05.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
40512026-03-24T21:02:05.740ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
40522026-03-24T21:02:05.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
40532026-03-24T21:02:05.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
40542026-03-24T21:02:05.743ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
40552026-03-24T21:02:05.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
40562026-03-24T21:02:05.745ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40572026-03-24T21:02:05.749ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
40582026-03-24T21:02:05.749ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
40592026-03-24T21:02:05.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
40602026-03-24T21:02:05.750ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40612026-03-24T21:02:05.750ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 196.314724ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
40622026-03-24T21:02:05.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
40632026-03-24T21:02:05.754ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40642026-03-24T21:02:05.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40652026-03-24T21:02:05.766ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
40662026-03-24T21:02:05.766ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40672026-03-24T21:02:05.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40682026-03-24T21:02:05.773ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
40692026-03-24T21:02:05.773ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
40702026-03-24T21:02:05.773ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 218.80111ms iteration = 6 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
40712026-03-24T21:02:05.775ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
40722026-03-24T21:02:05.775ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
40732026-03-24T21:02:05.775ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
40742026-03-24T21:02:05.775ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
40752026-03-24T21:02:05.776ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 221.336818ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40762026-03-24T21:02:05.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
40772026-03-24T21:02:05.788ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40782026-03-24T21:02:05.792ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
40792026-03-24T21:02:05.792ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
40802026-03-24T21:02:05.792ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40812026-03-24T21:02:05.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
40822026-03-24T21:02:05.793ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40832026-03-24T21:02:05.793ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
40842026-03-24T21:02:05.793ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 238.778639ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
40852026-03-24T21:02:05.795ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40862026-03-24T21:02:05.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40872026-03-24T21:02:05.797ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40882026-03-24T21:02:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
40892026-03-24T21:02:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = TargetReleaseConfig result = Ok(())
40902026-03-24T21:02:05.799ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
40912026-03-24T21:02:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
40922026-03-24T21:02:05.800ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
40932026-03-24T21:02:05.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
40942026-03-24T21:02:05.800ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Fleet result = Ok(())
40952026-03-24T21:02:05.800ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
40962026-03-24T21:02:05.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
40972026-03-24T21:02:05.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
40982026-03-24T21:02:05.803ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
40992026-03-24T21:02:05.803ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
41002026-03-24T21:02:05.803ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 271.030279ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
41012026-03-24T21:02:05.805ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41022026-03-24T21:02:05.806ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
41032026-03-24T21:02:05.806ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41042026-03-24T21:02:05.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41052026-03-24T21:02:05.807ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
41062026-03-24T21:02:05.807ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
41072026-03-24T21:02:05.807ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 231.773209ms iteration = 21 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
41082026-03-24T21:02:05.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41092026-03-24T21:02:05.809ZINFOnexus (BackgroundTasks): tuf_repo_prune decision background_task = tuf_repo_pruner file = nexus/src/app/background/tasks/tuf_repo_pruner.rs:153 status = TufRepoPrunerDetails { nkeep_recent_releases: 3, nkeep_recent_uploads: 2, repos_keep_target_release: {}, repos_keep_recent_uploads: {}, repo_prune: None, other_repos_eligible_to_prune: {}, warnings: [] }
41102026-03-24T21:02:05.809ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_repo_pruner elapsed = 255.095789ms iteration = 3 status = {"nkeep_recent_releases":3,"nkeep_recent_uploads":2,"other_repos_eligible_to_prune":[],"repo_prune":null,"repos_keep_recent_uploads":[],"repos_keep_target_release":[],"status":"enabled","warnings":[]}
41112026-03-24T21:02:05.810ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
41122026-03-24T21:02:05.810ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
41132026-03-24T21:02:05.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
41142026-03-24T21:02:05.812ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
41152026-03-24T21:02:05.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
41162026-03-24T21:02:05.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
41172026-03-24T21:02:05.814ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
41182026-03-24T21:02:05.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
41192026-03-24T21:02:05.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
41202026-03-24T21:02:05.817ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
41212026-03-24T21:02:05.817ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 271.588364ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
41222026-03-24T21:02:05.823ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "af461a5b-0734-4d42-b824-1b5ee8b6562b", "date": "Tue, 24 Mar 2026 21:02:05 GMT"} })
41232026-03-24T21:02:05.823ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
41242026-03-24T21:02:05.823ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
41252026-03-24T21:02:05.823ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 247.936526ms iteration = 6 status = {"errors":[],"probes_by_sled":{"b4293d4f-3308-4dfa-b261-a103215913e4":0}}
41262026-03-24T21:02:05.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41272026-03-24T21:02:05.828ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41282026-03-24T21:02:05.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41292026-03-24T21:02:05.832ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
41302026-03-24T21:02:05.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
41312026-03-24T21:02:05.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
41322026-03-24T21:02:05.834ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 280.028201ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
41332026-03-24T21:02:05.835ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
41342026-03-24T21:02:05.835ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
41352026-03-24T21:02:05.836ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
41362026-03-24T21:02:05.836ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
41372026-03-24T21:02:05.836ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
41382026-03-24T21:02:05.836ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
41392026-03-24T21:02:05.836ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
41402026-03-24T21:02:05.836ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41412026-03-24T21:02:05.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41422026-03-24T21:02:05.837ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
41432026-03-24T21:02:05.837ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
41442026-03-24T21:02:05.837ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41452026-03-24T21:02:05.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41462026-03-24T21:02:05.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
41472026-03-24T21:02:05.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41482026-03-24T21:02:05.838ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41492026-03-24T21:02:05.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
41502026-03-24T21:02:05.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41512026-03-24T21:02:05.840ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
41522026-03-24T21:02:05.840ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
41532026-03-24T21:02:05.841ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
41542026-03-24T21:02:05.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
41552026-03-24T21:02:05.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
41562026-03-24T21:02:05.842ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41572026-03-24T21:02:05.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41582026-03-24T21:02:05.842ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41592026-03-24T21:02:05.843ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
41602026-03-24T21:02:05.843ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 311.397147ms iteration = 10 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"29a1b98a-4509-4d5f-a750-afad7bceeaaa","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
41612026-03-24T21:02:05.845ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
41622026-03-24T21:02:05.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
41632026-03-24T21:02:05.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
41642026-03-24T21:02:05.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41652026-03-24T21:02:05.849ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
41662026-03-24T21:02:05.849ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
41672026-03-24T21:02:05.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
41682026-03-24T21:02:05.851ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
41692026-03-24T21:02:05.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
41702026-03-24T21:02:05.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
41712026-03-24T21:02:05.851ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
41722026-03-24T21:02:05.851ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41732026-03-24T21:02:05.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41742026-03-24T21:02:05.855ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41752026-03-24T21:02:05.855ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
41762026-03-24T21:02:05.855ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
41772026-03-24T21:02:05.855ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2026-03-24 20:57:05.047625 UTC dns_group = internal generation = 1
41782026-03-24T21:02:05.855ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 340.015998ms iteration = 6 status = {"generation":1}
41792026-03-24T21:02:05.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41802026-03-24T21:02:05.860ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41812026-03-24T21:02:05.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41822026-03-24T21:02:05.864ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41832026-03-24T21:02:05.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41842026-03-24T21:02:05.866ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41852026-03-24T21:02:05.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
41862026-03-24T21:02:05.867ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41872026-03-24T21:02:05.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41882026-03-24T21:02:05.868ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
41892026-03-24T21:02:05.868ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 336.541905ms iteration = 16 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
41902026-03-24T21:02:05.868ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41912026-03-24T21:02:05.871ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41922026-03-24T21:02:05.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
41932026-03-24T21:02:05.872ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
41942026-03-24T21:02:05.872ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
41952026-03-24T21:02:05.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
41962026-03-24T21:02:05.872ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
41972026-03-24T21:02:05.872ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 340.305117ms iteration = 31 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
41982026-03-24T21:02:05.873ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
41992026-03-24T21:02:05.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42002026-03-24T21:02:05.875ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42012026-03-24T21:02:05.881ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
42022026-03-24T21:02:05.881ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
42032026-03-24T21:02:05.881ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 326.638428ms iteration = 11 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
42042026-03-24T21:02:05.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
42052026-03-24T21:02:05.881ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42062026-03-24T21:02:05.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42072026-03-24T21:02:05.883ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42082026-03-24T21:02:05.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42092026-03-24T21:02:05.887ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42102026-03-24T21:02:05.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42112026-03-24T21:02:05.890ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
42122026-03-24T21:02:05.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
42132026-03-24T21:02:05.892ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
42142026-03-24T21:02:05.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
42152026-03-24T21:02:05.893ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42162026-03-24T21:02:05.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42172026-03-24T21:02:05.896ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
42182026-03-24T21:02:05.896ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42192026-03-24T21:02:05.896ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42202026-03-24T21:02:05.896ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9abab10f-c279-438b-85fc-0a02e09d5dfe", "content-length": "372", "date": "Tue, 24 Mar 2026 21:02:05 GMT"} })
42212026-03-24T21:02:05.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42222026-03-24T21:02:05.896ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42232026-03-24T21:02:05.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42242026-03-24T21:02:05.902ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42252026-03-24T21:02:05.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42262026-03-24T21:02:05.907ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42272026-03-24T21:02:05.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
42282026-03-24T21:02:05.909ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42292026-03-24T21:02:05.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42302026-03-24T21:02:05.912ZINFOnexus (BackgroundTasks): Loaded 0 active trust quorum configurations from database background_task = trust_quorum_manager bgtask = TrustQuorumManager file = nexus/src/app/background/tasks/trust_quorum.rs:83
42312026-03-24T21:02:05.912ZDEBGnexus (BackgroundTasks): activation complete background_task = trust_quorum_manager elapsed = 337.371968ms iteration = 6 status = {"PerRackStatus":{"errors":[],"statuses":[]}}
42322026-03-24T21:02:05.912ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42332026-03-24T21:02:05.922ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
42342026-03-24T21:02:05.922ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42352026-03-24T21:02:05.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42362026-03-24T21:02:05.930ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42372026-03-24T21:02:05.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42382026-03-24T21:02:05.934ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42392026-03-24T21:02:05.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42402026-03-24T21:02:05.937ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42412026-03-24T21:02:05.939ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
42422026-03-24T21:02:05.939ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
42432026-03-24T21:02:05.940ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42442026-03-24T21:02:05.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
42452026-03-24T21:02:05.942ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42462026-03-24T21:02:05.944ZDEBGnexus (BackgroundTasks): listed attached subnets background_task = attached_subnet_manager n_subnets = 0
42472026-03-24T21:02:05.945ZDEBGnexus (BackgroundTasks): activation complete background_task = attached_subnet_manager elapsed = 369.568505ms iteration = 6 status = {"db_error":null,"dendrite":{"switch0":{"errors":[],"n_subnets_added":0,"n_subnets_removed":0,"n_total_subnets":0}},"sled":{}}
42482026-03-24T21:02:05.947ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
42492026-03-24T21:02:05.947ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42502026-03-24T21:02:05.947ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
42512026-03-24T21:02:05.947ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42522026-03-24T21:02:05.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
42532026-03-24T21:02:05.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42542026-03-24T21:02:05.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
42552026-03-24T21:02:05.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
42562026-03-24T21:02:05.950ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
42572026-03-24T21:02:05.950ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
42582026-03-24T21:02:05.950ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2026-03-24 20:57:07.125641 UTC dns_group = external generation = 2
42592026-03-24T21:02:05.950ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 430.748265ms iteration = 6 status = {"generation":2}
42602026-03-24T21:02:05.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
42612026-03-24T21:02:05.953ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42622026-03-24T21:02:05.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
42632026-03-24T21:02:05.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42642026-03-24T21:02:05.955ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42652026-03-24T21:02:05.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42662026-03-24T21:02:05.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42672026-03-24T21:02:05.958ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42682026-03-24T21:02:05.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42692026-03-24T21:02:05.959ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
42702026-03-24T21:02:05.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
42712026-03-24T21:02:05.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
42722026-03-24T21:02:05.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
42732026-03-24T21:02:05.961ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42742026-03-24T21:02:05.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
42752026-03-24T21:02:05.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42762026-03-24T21:02:05.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42772026-03-24T21:02:05.965ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42782026-03-24T21:02:05.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42792026-03-24T21:02:05.966ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42802026-03-24T21:02:05.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42812026-03-24T21:02:05.967ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
42822026-03-24T21:02:05.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
42832026-03-24T21:02:05.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
42842026-03-24T21:02:05.968ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
42852026-03-24T21:02:05.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
42862026-03-24T21:02:05.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42872026-03-24T21:02:05.970ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42882026-03-24T21:02:05.972ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 417.804256ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
42892026-03-24T21:02:05.973ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2026-03-24T20:56:59.957750Z, time_modified: 2026-03-24T20:56:59.957750Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }) }
42902026-03-24T21:02:05.973ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = DeployNexusRecords step id = Ensure
42912026-03-24T21:02:05.973ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 453.287122ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]}},"default_endpoint":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
42922026-03-24T21:02:05.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42932026-03-24T21:02:05.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
42942026-03-24T21:02:05.973ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
42952026-03-24T21:02:05.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
42962026-03-24T21:02:05.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
42972026-03-24T21:02:05.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42982026-03-24T21:02:05.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
42992026-03-24T21:02:05.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43002026-03-24T21:02:05.978ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
43012026-03-24T21:02:05.978ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43022026-03-24T21:02:05.978ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
43032026-03-24T21:02:05.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
43042026-03-24T21:02:05.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43052026-03-24T21:02:05.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
43062026-03-24T21:02:05.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
43072026-03-24T21:02:05.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
43082026-03-24T21:02:05.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43092026-03-24T21:02:05.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43102026-03-24T21:02:05.985ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43112026-03-24T21:02:05.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43122026-03-24T21:02:05.989ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
43132026-03-24T21:02:05.990ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43142026-03-24T21:02:05.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43152026-03-24T21:02:05.991ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
43162026-03-24T21:02:05.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43172026-03-24T21:02:05.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43182026-03-24T21:02:05.993ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43192026-03-24T21:02:05.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43202026-03-24T21:02:05.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43212026-03-24T21:02:05.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43222026-03-24T21:02:05.997ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
43232026-03-24T21:02:05.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
43242026-03-24T21:02:06.000ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
43252026-03-24T21:02:06.000ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
43262026-03-24T21:02:06.000ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43272026-03-24T21:02:06.000ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
43282026-03-24T21:02:06.000ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43292026-03-24T21:02:06.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43302026-03-24T21:02:06.000ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "20718bab-1d0f-4e1c-9f4d-189866362c57", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
43312026-03-24T21:02:06.000ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43322026-03-24T21:02:06.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43332026-03-24T21:02:06.001ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
43342026-03-24T21:02:06.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43352026-03-24T21:02:06.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43362026-03-24T21:02:06.003ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
43372026-03-24T21:02:06.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43382026-03-24T21:02:06.003ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
43392026-03-24T21:02:06.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43402026-03-24T21:02:06.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
43412026-03-24T21:02:06.003ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43422026-03-24T21:02:06.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43432026-03-24T21:02:06.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43442026-03-24T21:02:06.005ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
43452026-03-24T21:02:06.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
43462026-03-24T21:02:06.008ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43472026-03-24T21:02:06.008ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43482026-03-24T21:02:06.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43492026-03-24T21:02:06.009ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
43502026-03-24T21:02:06.009ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43512026-03-24T21:02:06.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43522026-03-24T21:02:06.011ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
43532026-03-24T21:02:06.011ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
43542026-03-24T21:02:06.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
43552026-03-24T21:02:06.012ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
43562026-03-24T21:02:06.012ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
43572026-03-24T21:02:06.012ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
43582026-03-24T21:02:06.012ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43592026-03-24T21:02:06.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43602026-03-24T21:02:06.013ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
43612026-03-24T21:02:06.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
43622026-03-24T21:02:06.013ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "4a1230f6-b35f-4403-9b14-c535792fa9f5", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
43632026-03-24T21:02:06.013ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 459.249673ms iteration = 16 status = {}
43642026-03-24T21:02:06.013ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Dependency
43652026-03-24T21:02:06.014ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
43662026-03-24T21:02:06.014ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43672026-03-24T21:02:06.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43682026-03-24T21:02:06.014ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43692026-03-24T21:02:06.015ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43702026-03-24T21:02:06.016ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43712026-03-24T21:02:06.018ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
43722026-03-24T21:02:06.018ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
43732026-03-24T21:02:06.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
43742026-03-24T21:02:06.019ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
43752026-03-24T21:02:06.019ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
43762026-03-24T21:02:06.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
43772026-03-24T21:02:06.020ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
43782026-03-24T21:02:06.020ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43792026-03-24T21:02:06.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43802026-03-24T21:02:06.021ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
43812026-03-24T21:02:06.021ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
43822026-03-24T21:02:06.021ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43832026-03-24T21:02:06.022ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "cc622981-0e31-4c5b-9ee5-ac2faef42ec3", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
43842026-03-24T21:02:06.022ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
43852026-03-24T21:02:06.023ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f26a56c-d67f-4779-8619-b3d6fa588751", "content-length": "26", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
43862026-03-24T21:02:06.023ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:680 sled = http://[fd00:1122:3344:101::1]:12345
43872026-03-24T21:02:06.023ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 468.580935ms iteration = 2 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":2,"put_config_err":0,"put_config_ok":2,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"b4293d4f-3308-4dfa-b261-a103215913e4","time":"2026-03-24T20:57:06.273272074Z"},{"operation":"list","target_sled":"b4293d4f-3308-4dfa-b261-a103215913e4","time":"2026-03-24T20:57:06.274254260Z"},{"generation":1,"operation":"put_config","target_sled":"b4293d4f-3308-4dfa-b261-a103215913e4","time":"2026-03-24T21:02:06.022637029Z"},{"operation":"list","target_sled":"b4293d4f-3308-4dfa-b261-a103215913e4","time":"2026-03-24T21:02:06.023033189Z"}]}
43882026-03-24T21:02:06.024ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
43892026-03-24T21:02:06.024ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43902026-03-24T21:02:06.024ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
43912026-03-24T21:02:06.024ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
43922026-03-24T21:02:06.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
43932026-03-24T21:02:06.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43942026-03-24T21:02:06.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43952026-03-24T21:02:06.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43962026-03-24T21:02:06.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43972026-03-24T21:02:06.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43982026-03-24T21:02:06.044ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
43992026-03-24T21:02:06.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44002026-03-24T21:02:06.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44012026-03-24T21:02:06.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44022026-03-24T21:02:06.049ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 517.289359ms iteration = 19 status = {"physical_disks_added":0}
44032026-03-24T21:02:06.052ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44042026-03-24T21:02:06.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44052026-03-24T21:02:06.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44062026-03-24T21:02:06.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44072026-03-24T21:02:06.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44082026-03-24T21:02:06.057ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = SledAgent step id = Ensure
44092026-03-24T21:02:06.058ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
44102026-03-24T21:02:06.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44112026-03-24T21:02:06.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44122026-03-24T21:02:06.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44132026-03-24T21:02:06.064ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "4b075ff3-5c58-4662-bbfc-d32b9d288d32", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} }) sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
44142026-03-24T21:02:06.064ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = FirewallRules step id = Ensure
44152026-03-24T21:02:06.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44162026-03-24T21:02:06.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44172026-03-24T21:02:06.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44182026-03-24T21:02:06.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44192026-03-24T21:02:06.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44202026-03-24T21:02:06.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44212026-03-24T21:02:06.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44222026-03-24T21:02:06.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44232026-03-24T21:02:06.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44242026-03-24T21:02:06.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44252026-03-24T21:02:06.071ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44262026-03-24T21:02:06.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44272026-03-24T21:02:06.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44282026-03-24T21:02:06.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44292026-03-24T21:02:06.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44302026-03-24T21:02:06.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44312026-03-24T21:02:06.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44322026-03-24T21:02:06.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44332026-03-24T21:02:06.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44342026-03-24T21:02:06.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44352026-03-24T21:02:06.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44362026-03-24T21:02:06.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44372026-03-24T21:02:06.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44382026-03-24T21:02:06.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44392026-03-24T21:02:06.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
44402026-03-24T21:02:06.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44412026-03-24T21:02:06.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44422026-03-24T21:02:06.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44432026-03-24T21:02:06.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44442026-03-24T21:02:06.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44452026-03-24T21:02:06.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44462026-03-24T21:02:06.090ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
44472026-03-24T21:02:06.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44482026-03-24T21:02:06.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44492026-03-24T21:02:06.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
44502026-03-24T21:02:06.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44512026-03-24T21:02:06.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44522026-03-24T21:02:06.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44532026-03-24T21:02:06.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44542026-03-24T21:02:06.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44552026-03-24T21:02:06.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44562026-03-24T21:02:06.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44572026-03-24T21:02:06.103ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
44582026-03-24T21:02:06.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44592026-03-24T21:02:06.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44602026-03-24T21:02:06.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44612026-03-24T21:02:06.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44622026-03-24T21:02:06.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44632026-03-24T21:02:06.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44642026-03-24T21:02:06.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44652026-03-24T21:02:06.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44662026-03-24T21:02:06.113ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44672026-03-24T21:02:06.113ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44682026-03-24T21:02:06.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44692026-03-24T21:02:06.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44702026-03-24T21:02:06.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44712026-03-24T21:02:06.117ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
44722026-03-24T21:02:06.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44732026-03-24T21:02:06.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44742026-03-24T21:02:06.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44752026-03-24T21:02:06.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44762026-03-24T21:02:06.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44772026-03-24T21:02:06.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44782026-03-24T21:02:06.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44792026-03-24T21:02:06.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44802026-03-24T21:02:06.121ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e9527e99-34db-48dd-b735-c21d284ce627", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
44812026-03-24T21:02:06.121ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
44822026-03-24T21:02:06.121ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44832026-03-24T21:02:06.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44842026-03-24T21:02:06.122ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:118 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
44852026-03-24T21:02:06.122ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 567.998787ms iteration = 6 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
44862026-03-24T21:02:06.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44872026-03-24T21:02:06.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44882026-03-24T21:02:06.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
44892026-03-24T21:02:06.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44902026-03-24T21:02:06.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44912026-03-24T21:02:06.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
44922026-03-24T21:02:06.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44932026-03-24T21:02:06.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44942026-03-24T21:02:06.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44952026-03-24T21:02:06.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44962026-03-24T21:02:06.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44972026-03-24T21:02:06.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44982026-03-24T21:02:06.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44992026-03-24T21:02:06.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45002026-03-24T21:02:06.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45012026-03-24T21:02:06.138ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
45022026-03-24T21:02:06.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45032026-03-24T21:02:06.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45042026-03-24T21:02:06.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45052026-03-24T21:02:06.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45062026-03-24T21:02:06.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45072026-03-24T21:02:06.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45082026-03-24T21:02:06.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45092026-03-24T21:02:06.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45102026-03-24T21:02:06.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45112026-03-24T21:02:06.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
45122026-03-24T21:02:06.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45132026-03-24T21:02:06.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45142026-03-24T21:02:06.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45152026-03-24T21:02:06.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45162026-03-24T21:02:06.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45172026-03-24T21:02:06.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45182026-03-24T21:02:06.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45192026-03-24T21:02:06.160ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
45202026-03-24T21:02:06.160ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
45212026-03-24T21:02:06.160ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
45222026-03-24T21:02:06.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45232026-03-24T21:02:06.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45242026-03-24T21:02:06.161ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b631c863-bdc9-42c1-8244-d3f4bbd47fdc", "content-length": "372", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
45252026-03-24T21:02:06.161ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45262026-03-24T21:02:06.161ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45272026-03-24T21:02:06.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45282026-03-24T21:02:06.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
45292026-03-24T21:02:06.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45302026-03-24T21:02:06.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45312026-03-24T21:02:06.163ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45322026-03-24T21:02:06.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45332026-03-24T21:02:06.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45342026-03-24T21:02:06.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45352026-03-24T21:02:06.167ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
45362026-03-24T21:02:06.167ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45372026-03-24T21:02:06.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
45382026-03-24T21:02:06.170ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45392026-03-24T21:02:06.174ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
45402026-03-24T21:02:06.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45412026-03-24T21:02:06.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45422026-03-24T21:02:06.174ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
45432026-03-24T21:02:06.174ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
45442026-03-24T21:02:06.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
45452026-03-24T21:02:06.176ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
45462026-03-24T21:02:06.176ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
45472026-03-24T21:02:06.177ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "afb0b392-6817-4b19-aaaa-35ec04ad0b57", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
45482026-03-24T21:02:06.177ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45492026-03-24T21:02:06.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45502026-03-24T21:02:06.179ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
45512026-03-24T21:02:06.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45522026-03-24T21:02:06.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45532026-03-24T21:02:06.179ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb6c89ea-9086-4d9e-850e-75150be8ed60", "content-length": "691", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
45542026-03-24T21:02:06.179ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
45552026-03-24T21:02:06.179ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
45562026-03-24T21:02:06.179ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
45572026-03-24T21:02:06.179ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 647.384646ms iteration = 11 status = {}
45582026-03-24T21:02:06.180ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45592026-03-24T21:02:06.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45602026-03-24T21:02:06.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45612026-03-24T21:02:06.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45622026-03-24T21:02:06.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45632026-03-24T21:02:06.184ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45642026-03-24T21:02:06.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45652026-03-24T21:02:06.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45662026-03-24T21:02:06.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45672026-03-24T21:02:06.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45682026-03-24T21:02:06.187ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
45692026-03-24T21:02:06.187ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
45702026-03-24T21:02:06.188ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7c9bebd8-2176-43c2-8a6b-b1516d501f8d", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
45712026-03-24T21:02:06.188ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 174.556867ms iteration = 17 status = {}
45722026-03-24T21:02:06.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45732026-03-24T21:02:06.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45742026-03-24T21:02:06.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45752026-03-24T21:02:06.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45762026-03-24T21:02:06.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45772026-03-24T21:02:06.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45782026-03-24T21:02:06.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45792026-03-24T21:02:06.199ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45802026-03-24T21:02:06.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45812026-03-24T21:02:06.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45822026-03-24T21:02:06.204ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
45832026-03-24T21:02:06.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45842026-03-24T21:02:06.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45852026-03-24T21:02:06.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45862026-03-24T21:02:06.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45872026-03-24T21:02:06.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45882026-03-24T21:02:06.207ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45892026-03-24T21:02:06.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45902026-03-24T21:02:06.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45912026-03-24T21:02:06.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45922026-03-24T21:02:06.209ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 677.528659ms iteration = 11 status = {}
45932026-03-24T21:02:06.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45942026-03-24T21:02:06.213ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
45952026-03-24T21:02:06.213ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
45962026-03-24T21:02:06.213ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 0e46a789-e46e-4abf-9dfe-137c8e53d65a parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
45972026-03-24T21:02:06.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45982026-03-24T21:02:06.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45992026-03-24T21:02:06.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46002026-03-24T21:02:06.215ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
46012026-03-24T21:02:06.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46022026-03-24T21:02:06.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46032026-03-24T21:02:06.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
46042026-03-24T21:02:06.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
46052026-03-24T21:02:06.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
46062026-03-24T21:02:06.222ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
46072026-03-24T21:02:06.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46082026-03-24T21:02:06.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46092026-03-24T21:02:06.223ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
46102026-03-24T21:02:06.223ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
46112026-03-24T21:02:06.223ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 691.115045ms iteration = 16 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
46122026-03-24T21:02:06.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46132026-03-24T21:02:06.225ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
46142026-03-24T21:02:06.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46152026-03-24T21:02:06.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46162026-03-24T21:02:06.227ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
46172026-03-24T21:02:06.227ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
46182026-03-24T21:02:06.227ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
46192026-03-24T21:02:06.248ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
46202026-03-24T21:02:06.248ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
46212026-03-24T21:02:06.249ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46222026-03-24T21:02:06.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46232026-03-24T21:02:06.253ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
46242026-03-24T21:02:06.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
46252026-03-24T21:02:06.255ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46262026-03-24T21:02:06.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
46272026-03-24T21:02:06.271ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
46282026-03-24T21:02:06.271ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "f967e6f9-61b1-45ae-b508-fde27362214f", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
46292026-03-24T21:02:06.272ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:506
46302026-03-24T21:02:06.272ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 717.406074ms
46312026-03-24T21:02:06.272ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 717.423245ms iteration = 2 status = {}
46322026-03-24T21:02:06.292ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
46332026-03-24T21:02:06.292ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
46342026-03-24T21:02:06.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46352026-03-24T21:02:06.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46362026-03-24T21:02:06.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46372026-03-24T21:02:06.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46382026-03-24T21:02:06.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46392026-03-24T21:02:06.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
46402026-03-24T21:02:06.300ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
46412026-03-24T21:02:06.300ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "63442c48-5059-4100-86dc-0d07e16267fe", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
46422026-03-24T21:02:06.300ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
46432026-03-24T21:02:06.300ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = Dns step id = Ensure
46442026-03-24T21:02:06.300ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
46452026-03-24T21:02:06.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46462026-03-24T21:02:06.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46472026-03-24T21:02:06.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46482026-03-24T21:02:06.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46492026-03-24T21:02:06.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46502026-03-24T21:02:06.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
46512026-03-24T21:02:06.355ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
46522026-03-24T21:02:06.355ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
46532026-03-24T21:02:06.355ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
46542026-03-24T21:02:06.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46552026-03-24T21:02:06.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46562026-03-24T21:02:06.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46572026-03-24T21:02:06.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
46582026-03-24T21:02:06.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46592026-03-24T21:02:06.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46602026-03-24T21:02:06.403ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
46612026-03-24T21:02:06.403ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
46622026-03-24T21:02:06.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46632026-03-24T21:02:06.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46642026-03-24T21:02:06.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46652026-03-24T21:02:06.409ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
46662026-03-24T21:02:06.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46672026-03-24T21:02:06.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46682026-03-24T21:02:06.412ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
46692026-03-24T21:02:06.412ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
46702026-03-24T21:02:06.412ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
46712026-03-24T21:02:06.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46722026-03-24T21:02:06.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46732026-03-24T21:02:06.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46742026-03-24T21:02:06.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46752026-03-24T21:02:06.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46762026-03-24T21:02:06.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
46772026-03-24T21:02:06.421ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
46782026-03-24T21:02:06.421ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
46792026-03-24T21:02:06.421ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
46802026-03-24T21:02:06.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46812026-03-24T21:02:06.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46822026-03-24T21:02:06.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46832026-03-24T21:02:06.426ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
46842026-03-24T21:02:06.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46852026-03-24T21:02:06.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46862026-03-24T21:02:06.474ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
46872026-03-24T21:02:06.474ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
46882026-03-24T21:02:06.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46892026-03-24T21:02:06.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46902026-03-24T21:02:06.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46912026-03-24T21:02:06.479ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
46922026-03-24T21:02:06.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46932026-03-24T21:02:06.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46942026-03-24T21:02:06.481ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
46952026-03-24T21:02:06.481ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
46962026-03-24T21:02:06.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46972026-03-24T21:02:06.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46982026-03-24T21:02:06.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46992026-03-24T21:02:06.484ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
47002026-03-24T21:02:06.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47012026-03-24T21:02:06.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47022026-03-24T21:02:06.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47032026-03-24T21:02:06.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47042026-03-24T21:02:06.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47052026-03-24T21:02:06.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47062026-03-24T21:02:06.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47072026-03-24T21:02:06.493ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
47082026-03-24T21:02:06.494ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
47092026-03-24T21:02:06.494ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
47102026-03-24T21:02:06.494ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = OmicronZones step id = Cleanup
47112026-03-24T21:02:06.494ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = OmicronZones step id = Cleanup
47122026-03-24T21:02:06.494ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = PhysicalDisks step id = Cleanup
47132026-03-24T21:02:06.494ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = Clickhouse step id = Ensure
47142026-03-24T21:02:06.494ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = Clickhouse step id = Ensure
47152026-03-24T21:02:06.495ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
47162026-03-24T21:02:06.499ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "90ce0eef-5846-4bf2-b8f0-9a4e72c7c5e2", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
47172026-03-24T21:02:06.499ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = SupportBundles step id = Cleanup
47182026-03-24T21:02:06.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47192026-03-24T21:02:06.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47202026-03-24T21:02:06.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47212026-03-24T21:02:06.503ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
47222026-03-24T21:02:06.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47232026-03-24T21:02:06.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47242026-03-24T21:02:06.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47252026-03-24T21:02:06.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47262026-03-24T21:02:06.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47272026-03-24T21:02:06.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
47282026-03-24T21:02:06.558ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = OmicronZones step id = Cleanup
47292026-03-24T21:02:06.558ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
47302026-03-24T21:02:06.558ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
47312026-03-24T21:02:06.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47322026-03-24T21:02:06.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47332026-03-24T21:02:06.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47342026-03-24T21:02:06.562ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
47352026-03-24T21:02:06.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47362026-03-24T21:02:06.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47372026-03-24T21:02:06.610ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
47382026-03-24T21:02:06.610ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9e1c09c1-3960-4ecf-a904-19e137699ab3 (blueprint), false)
47392026-03-24T21:02:06.610ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = Cockroach step id = Ensure
47402026-03-24T21:02:06.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47412026-03-24T21:02:06.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47422026-03-24T21:02:06.637ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
47432026-03-24T21:02:06.637ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5456b416-21da-4928-9a0d-d5e1d49d7233 step component = MgsUpdates step id = Ensure
47442026-03-24T21:02:06.638ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 17 reason = Dependency
47452026-03-24T21:02:06.638ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 8 reason = Dependency
47462026-03-24T21:02:06.638ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47472026-03-24T21:02:06.638ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47482026-03-24T21:02:06.638ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.117326858s iteration = 7 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3130,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":93474910,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":93474861,"secs":0}},"event_index":1,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":93501632,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":358835806,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":358835766,"secs":0}},"event_index":2,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":452357888,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":84481528,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":84481488,"secs":0}},"event_index":3,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":536851196,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6690550,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":6690520,"secs":0}},"event_index":4,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":543564087,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":236364243,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":236364203,"secs":0}},"event_index":5,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":779938341,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":193508057,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":193508017,"secs":0}},"event_index":6,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":973452879,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5070,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5040,"secs":0}},"event_index":7,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":973470179,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3320,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3280,"secs":0}},"event_index":8,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":973476119,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3430,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3390,"secs":0}},"event_index":9,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":973481049,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1610,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1570,"secs":0}},"event_index":10,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":973484219,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5525378,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":5525349,"secs":0}},"event_index":11,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":979014429,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":58883047,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":58883017,"secs":0}},"event_index":12,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":37904636,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":51614401,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":51614361,"secs":0}},"event_index":13,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":89533238,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":27465700,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":27465660,"secs":0}},"event_index":14,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":117005319,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5820,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5780,"secs":0}},"event_index":15,"execution_id":"b391752c-f15f-4b63-a6bf-750a95c8c0d9","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":117020450,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3"}
47492026-03-24T21:02:06.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47502026-03-24T21:02:06.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
47512026-03-24T21:02:06.639ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47522026-03-24T21:02:06.639ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47532026-03-24T21:02:06.642ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
47542026-03-24T21:02:06.642ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47552026-03-24T21:02:06.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47562026-03-24T21:02:06.642ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
47572026-03-24T21:02:06.642ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47582026-03-24T21:02:06.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
47592026-03-24T21:02:06.679ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(e0be8d7c-c2e1-4aab-8e77-8dbb701a026c, 0), (8e786f99-51a8-4eb4-98a7-4302327a1608, 0), (1b4f62ac-a6d9-4eb1-9a06-e4cde54a1439, 0), (29a1b98a-4509-4d5f-a750-afad7bceeaaa, 0)] collection_id = e0be8d7c-c2e1-4aab-8e77-8dbb701a026c
47602026-03-24T21:02:06.679ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47612026-03-24T21:02:06.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
47622026-03-24T21:02:06.686ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47632026-03-24T21:02:06.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47642026-03-24T21:02:06.687ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47652026-03-24T21:02:06.689ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
47662026-03-24T21:02:06.689ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
47672026-03-24T21:02:06.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
47682026-03-24T21:02:06.738ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
47692026-03-24T21:02:06.738ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 100.390238ms iteration = 17 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
47702026-03-24T21:02:06.821ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = e0be8d7c-c2e1-4aab-8e77-8dbb701a026c file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
47712026-03-24T21:02:06.821ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47722026-03-24T21:02:06.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
47732026-03-24T21:02:06.826ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(8e786f99-51a8-4eb4-98a7-4302327a1608, 0), (1b4f62ac-a6d9-4eb1-9a06-e4cde54a1439, 0), (29a1b98a-4509-4d5f-a750-afad7bceeaaa, 0)]
47742026-03-24T21:02:06.827ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1113, tv_nsec: 41759779 } })
47752026-03-24T21:02:06.829ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1345db7a-abc1-4a4c-a49c-e6c56102e521.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1920db37-87f5-4a96-b3ef-1db2eb59e9a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6b02d2eb-4373-4f12-8efc-4c0e3f4fa77b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("709c4704-f768-4640-a198-a23621ed6e11.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1113, tv_nsec: 43371583 } })
47762026-03-24T21:02:06.829ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
47772026-03-24T21:02:06.829ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
47782026-03-24T21:02:06.829ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
47792026-03-24T21:02:06.830ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "438ec03f-9c8a-474a-9260-3acedbba1f5c", "content-length": "476", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
47802026-03-24T21:02:06.830ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
47812026-03-24T21:02:06.830ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9bd12f0-b70b-4d26-96b9-ea411b8945df", "content-length": "734", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
47822026-03-24T21:02:06.830ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
47832026-03-24T21:02:06.830ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "383e0550-e78a-41ef-92e9-1bfeefd64595", "content-length": "111", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
47842026-03-24T21:02:06.830ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
47852026-03-24T21:02:06.831ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "47f9896e-94fa-4b92-b5f2-b70724c0183b", "content-length": "111", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
47862026-03-24T21:02:06.831ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
47872026-03-24T21:02:06.831ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "11d55b90-f1ce-447a-9c10-dddd9b61d120", "content-length": "163", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
47882026-03-24T21:02:06.831ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
47892026-03-24T21:02:06.832ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb958c90-6da3-4169-b5e1-9b516fe1b30b", "content-length": "163", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
47902026-03-24T21:02:06.832ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
47912026-03-24T21:02:06.832ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "242ce7ac-d1ca-4fd7-8079-3c621e479311", "content-length": "166", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
47922026-03-24T21:02:06.832ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
47932026-03-24T21:02:06.833ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8eaaeeb1-9bed-4653-b004-01674d677a8b", "content-length": "166", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
47942026-03-24T21:02:06.833ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
47952026-03-24T21:02:06.833ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "03b01498-12a0-448e-a6c6-4b3df43ae04f", "content-length": "702", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
47962026-03-24T21:02:06.833ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
47972026-03-24T21:02:06.833ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3a12a8a-caa7-4e1a-8342-d27dadf57b8b", "content-length": "727", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
47982026-03-24T21:02:06.833ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
47992026-03-24T21:02:06.833ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d81482d8-c69c-4c1f-89e8-d539a5a77468", "content-length": "729", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48002026-03-24T21:02:06.833ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
48012026-03-24T21:02:06.833ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c2ad894-cdea-45aa-8586-88a68334f071", "content-length": "728", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48022026-03-24T21:02:06.833ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
48032026-03-24T21:02:06.834ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9056426-efc7-4958-9515-a036d1e61638", "content-length": "722", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48042026-03-24T21:02:06.834ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
48052026-03-24T21:02:06.834ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca23b4ec-4e50-4720-9780-22385f6d8cd0", "content-length": "10", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48062026-03-24T21:02:06.834ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
48072026-03-24T21:02:06.834ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a400021-db4e-438f-97e5-a46200fcf78d", "content-length": "144", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48082026-03-24T21:02:06.834ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
48092026-03-24T21:02:06.834ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ea043e2-f273-4b8d-ad23-f90cd313b1e1", "content-length": "147", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48102026-03-24T21:02:06.834ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
48112026-03-24T21:02:06.835ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7cf2244-a599-48f9-a257-055d169a5859", "content-length": "109", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48122026-03-24T21:02:06.835ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
48132026-03-24T21:02:06.835ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "90d4bacc-d08c-4cc9-89e2-b7a39fbcd41b", "content-length": "109", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48142026-03-24T21:02:06.835ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
48152026-03-24T21:02:06.836ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdeee525-f692-44a9-81b3-83b0476cf6a7", "content-length": "163", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48162026-03-24T21:02:06.836ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
48172026-03-24T21:02:06.836ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9bc0a3f-738c-4271-a017-398490e9cb0b", "content-length": "163", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48182026-03-24T21:02:06.836ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
48192026-03-24T21:02:06.837ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "33d055ed-48c9-45cb-aba6-7c811e7166b9", "content-length": "166", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48202026-03-24T21:02:06.837ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
48212026-03-24T21:02:06.837ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "012fa4ba-c5f5-4b97-b1ac-a9c5eaed21f9", "content-length": "166", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48222026-03-24T21:02:06.837ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
48232026-03-24T21:02:06.837ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfdacd2d-9c10-4722-bb4c-3c4e16ca22a4", "content-length": "702", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48242026-03-24T21:02:06.837ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
48252026-03-24T21:02:06.837ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8009faa7-37dc-424f-88a3-f662219ccf1b", "content-length": "727", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48262026-03-24T21:02:06.837ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
48272026-03-24T21:02:06.838ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a8b7d8f-d7c9-4a37-b5ec-47dabbd573ca", "content-length": "729", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48282026-03-24T21:02:06.838ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
48292026-03-24T21:02:06.838ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0849462d-cbe0-4282-b296-e22b9a5cdeb3", "content-length": "728", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48302026-03-24T21:02:06.838ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
48312026-03-24T21:02:06.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
48322026-03-24T21:02:06.840ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48332026-03-24T21:02:06.842ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
48342026-03-24T21:02:06.842ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
48352026-03-24T21:02:06.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
48362026-03-24T21:02:06.844ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48372026-03-24T21:02:06.846ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
48382026-03-24T21:02:06.846ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
48392026-03-24T21:02:06.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
48402026-03-24T21:02:06.850ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
48412026-03-24T21:02:06.850ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
48422026-03-24T21:02:06.915ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a232a68-c57b-4b13-ba04-e5af423fc45e", "content-length": "110670", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48432026-03-24T21:02:06.917ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
48442026-03-24T21:02:06.917ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
48452026-03-24T21:02:06.917ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
48462026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
48472026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
48482026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
48492026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
48502026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
48512026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "50a21246-f35e-4224-9732-6b53d3c00d18", "content-length": "64", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48522026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c55ede9e-1962-4e9c-bf89-3b73adf582e6", "content-length": "64", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48532026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
48542026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
48552026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8308cd4a-3226-401d-b1c3-2a90514c690c", "content-length": "64", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48562026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e1e52ff-8d73-404b-a831-8520da00addd", "content-length": "64", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48572026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f927ab7-ff4e-44f9-a566-46c04f0a8b1f", "content-length": "64", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48582026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
48592026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
48602026-03-24T21:02:06.917ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
48612026-03-24T21:02:06.927ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "842e4ff8-0a4a-43ec-8d2e-7d9f82aefe9b", "content-length": "387979", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48622026-03-24T21:02:06.927ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "afe59e65-be96-41c4-82c8-2e4395a2a0f5", "content-length": "398510", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48632026-03-24T21:02:06.928ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "0df179f9-c871-4622-b897-485807ed43bc", "content-length": "393645", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48642026-03-24T21:02:06.928ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe31ece4-27fa-4bb1-9a84-4fbb72fad744", "content-length": "391521", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48652026-03-24T21:02:06.932ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7b684d5-cf9b-47c3-bceb-7064f495bae5", "content-length": "424244", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48662026-03-24T21:02:06.935ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
48672026-03-24T21:02:06.935ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
48682026-03-24T21:02:06.935ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
48692026-03-24T21:02:06.935ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
48702026-03-24T21:02:06.935ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
48712026-03-24T21:02:06.936ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 (omicron_zone)
48722026-03-24T21:02:06.936ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
48732026-03-24T21:02:06.940ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5f653bf-7384-42ed-a861-911e313af030", "content-length": "128", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48742026-03-24T21:02:06.940ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
48752026-03-24T21:02:06.942ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
48762026-03-24T21:02:06.942ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
48772026-03-24T21:02:06.943ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "92c108ed-12fa-42f8-bca7-52137180787d", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48782026-03-24T21:02:06.943ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
48792026-03-24T21:02:06.943ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
48802026-03-24T21:02:06.943ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
48812026-03-24T21:02:06.943ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "82c1530d-01d7-4a16-95ec-357d0fd4b4dc", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48822026-03-24T21:02:06.943ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
48832026-03-24T21:02:06.943ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
48842026-03-24T21:02:06.943ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
48852026-03-24T21:02:06.944ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "102383ea-16fd-4846-9ff9-ee184c778729", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:02:06 GMT"} })
48862026-03-24T21:02:06.944ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
48872026-03-24T21:02:06.944ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
48882026-03-24T21:02:06.944ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
48892026-03-24T21:02:06.944ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
48902026-03-24T21:02:06.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
48912026-03-24T21:02:06.946ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48922026-03-24T21:02:06.948ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
48932026-03-24T21:02:06.948ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
48942026-03-24T21:02:06.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
48952026-03-24T21:02:07.176ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 3d19abfd-8175-4fec-8eaa-cfb3a4c435c7 file = nexus/db-queries/src/db/datastore/inventory.rs:1757
48962026-03-24T21:02:07.176ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 3d19abfd-8175-4fec-8eaa-cfb3a4c435c7 time_started = 2026-03-24 21:02:06.829541 UTC
48972026-03-24T21:02:07.176ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 29 reason = Dependency
48982026-03-24T21:02:07.176ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
48992026-03-24T21:02:07.176ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 538.603233ms iteration = 8 status = {"collection_id":"3d19abfd-8175-4fec-8eaa-cfb3a4c435c7","time_done":"2026-03-24 21:02:06.944519 UTC","time_started":"2026-03-24 21:02:06.829541 UTC"}
49002026-03-24T21:02:07.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
49012026-03-24T21:02:07.178ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49022026-03-24T21:02:07.181ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
49032026-03-24T21:02:07.181ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
49042026-03-24T21:02:07.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
49052026-03-24T21:02:07.183ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
49062026-03-24T21:02:07.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
49072026-03-24T21:02:07.236ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 15 reason = Dependency
49082026-03-24T21:02:07.236ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 11 reason = Dependency
49092026-03-24T21:02:07.236ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 60.015218ms iteration = 29 status = {"Loaded":{"collection_id":"9789333b-5bf2-4c58-8e77-63169dbd8708","time_loaded":"2026-03-24T21:02:07.176561544Z","time_started":"2026-03-24T21:02:06.888816Z"}}
49102026-03-24T21:02:07.236ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
49112026-03-24T21:02:07.236ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Dependency
49122026-03-24T21:02:07.236ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 21.701\u{b5}s iteration = 15 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
49132026-03-24T21:02:07.236ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 17 reason = Dependency
49142026-03-24T21:02:07.236ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49152026-03-24T21:02:07.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49162026-03-24T21:02:07.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49172026-03-24T21:02:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49182026-03-24T21:02:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49192026-03-24T21:02:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49202026-03-24T21:02:07.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49212026-03-24T21:02:07.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49222026-03-24T21:02:07.238ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49232026-03-24T21:02:07.241ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
49242026-03-24T21:02:07.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49252026-03-24T21:02:07.241ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
49262026-03-24T21:02:07.241ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49272026-03-24T21:02:07.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49282026-03-24T21:02:07.242ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
49292026-03-24T21:02:07.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49302026-03-24T21:02:07.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49312026-03-24T21:02:07.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49322026-03-24T21:02:07.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49332026-03-24T21:02:07.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49342026-03-24T21:02:07.243ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49352026-03-24T21:02:07.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49362026-03-24T21:02:07.245ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49372026-03-24T21:02:07.245ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
49382026-03-24T21:02:07.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49392026-03-24T21:02:07.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49402026-03-24T21:02:07.246ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
49412026-03-24T21:02:07.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49422026-03-24T21:02:07.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49432026-03-24T21:02:07.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49442026-03-24T21:02:07.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49452026-03-24T21:02:07.247ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
49462026-03-24T21:02:07.247ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
49472026-03-24T21:02:07.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
49482026-03-24T21:02:07.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49492026-03-24T21:02:07.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49502026-03-24T21:02:07.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49512026-03-24T21:02:07.250ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
49522026-03-24T21:02:07.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49532026-03-24T21:02:07.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49542026-03-24T21:02:07.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49552026-03-24T21:02:07.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49562026-03-24T21:02:07.253ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
49572026-03-24T21:02:07.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49582026-03-24T21:02:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49592026-03-24T21:02:07.254ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
49602026-03-24T21:02:07.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49612026-03-24T21:02:07.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49622026-03-24T21:02:07.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49632026-03-24T21:02:07.256ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
49642026-03-24T21:02:07.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49652026-03-24T21:02:07.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49662026-03-24T21:02:07.257ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
49672026-03-24T21:02:07.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49682026-03-24T21:02:07.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49692026-03-24T21:02:07.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49702026-03-24T21:02:07.259ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
49712026-03-24T21:02:07.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49722026-03-24T21:02:07.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49732026-03-24T21:02:07.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49742026-03-24T21:02:07.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49752026-03-24T21:02:07.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49762026-03-24T21:02:07.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49772026-03-24T21:02:07.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49782026-03-24T21:02:07.264ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
49792026-03-24T21:02:07.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49802026-03-24T21:02:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49812026-03-24T21:02:07.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49822026-03-24T21:02:07.266ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
49832026-03-24T21:02:07.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49842026-03-24T21:02:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49852026-03-24T21:02:07.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49862026-03-24T21:02:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
49872026-03-24T21:02:07.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49882026-03-24T21:02:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49892026-03-24T21:02:07.270ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
49902026-03-24T21:02:07.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49912026-03-24T21:02:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49922026-03-24T21:02:07.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49932026-03-24T21:02:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49942026-03-24T21:02:07.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49952026-03-24T21:02:07.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49962026-03-24T21:02:07.273ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
49972026-03-24T21:02:07.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
49982026-03-24T21:02:07.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
49992026-03-24T21:02:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
50002026-03-24T21:02:07.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50012026-03-24T21:02:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50022026-03-24T21:02:07.275ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
50032026-03-24T21:02:07.275ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 39.039243ms iteration = 11 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"9789333b-5bf2-4c58-8e77-63169dbd8708","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
50042026-03-24T21:02:07.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50052026-03-24T21:02:07.279ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
50062026-03-24T21:02:07.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50072026-03-24T21:02:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50082026-03-24T21:02:07.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50092026-03-24T21:02:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50102026-03-24T21:02:07.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50112026-03-24T21:02:07.285ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
50122026-03-24T21:02:07.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50132026-03-24T21:02:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50142026-03-24T21:02:07.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50152026-03-24T21:02:07.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50162026-03-24T21:02:07.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50172026-03-24T21:02:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
50182026-03-24T21:02:07.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50192026-03-24T21:02:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50202026-03-24T21:02:07.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50212026-03-24T21:02:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50222026-03-24T21:02:07.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50232026-03-24T21:02:07.297ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 60.543272ms iteration = 20 status = {"physical_disks_added":0}
50242026-03-24T21:02:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
50252026-03-24T21:02:07.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50262026-03-24T21:02:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50272026-03-24T21:02:07.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50282026-03-24T21:02:07.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50292026-03-24T21:02:07.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50302026-03-24T21:02:07.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50312026-03-24T21:02:07.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50322026-03-24T21:02:07.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
50332026-03-24T21:02:07.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50342026-03-24T21:02:07.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50352026-03-24T21:02:07.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50362026-03-24T21:02:07.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
50372026-03-24T21:02:07.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50382026-03-24T21:02:07.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50392026-03-24T21:02:07.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50402026-03-24T21:02:07.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50412026-03-24T21:02:07.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50422026-03-24T21:02:07.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50432026-03-24T21:02:07.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50442026-03-24T21:02:07.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
50452026-03-24T21:02:07.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50462026-03-24T21:02:07.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50472026-03-24T21:02:07.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50482026-03-24T21:02:07.343ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
50492026-03-24T21:02:07.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50502026-03-24T21:02:07.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50512026-03-24T21:02:07.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50522026-03-24T21:02:07.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50532026-03-24T21:02:07.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50542026-03-24T21:02:07.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50552026-03-24T21:02:07.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50562026-03-24T21:02:07.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
50572026-03-24T21:02:07.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50582026-03-24T21:02:07.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50592026-03-24T21:02:07.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50602026-03-24T21:02:07.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50612026-03-24T21:02:07.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50622026-03-24T21:02:07.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
50632026-03-24T21:02:07.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50642026-03-24T21:02:07.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50652026-03-24T21:02:07.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50662026-03-24T21:02:07.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50672026-03-24T21:02:07.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50682026-03-24T21:02:07.363ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
50692026-03-24T21:02:07.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50702026-03-24T21:02:07.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50712026-03-24T21:02:07.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50722026-03-24T21:02:07.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50732026-03-24T21:02:07.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50742026-03-24T21:02:07.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
50752026-03-24T21:02:07.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50762026-03-24T21:02:07.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50772026-03-24T21:02:07.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50782026-03-24T21:02:07.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50792026-03-24T21:02:07.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50802026-03-24T21:02:07.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
50812026-03-24T21:02:07.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50822026-03-24T21:02:07.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50832026-03-24T21:02:07.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50842026-03-24T21:02:07.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50852026-03-24T21:02:07.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50862026-03-24T21:02:07.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
50872026-03-24T21:02:07.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50882026-03-24T21:02:07.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50892026-03-24T21:02:07.423ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
50902026-03-24T21:02:07.423ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
50912026-03-24T21:02:07.423ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 7490fe0d-3fa1-4d60-9620-08fbc2ae8b8d parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
50922026-03-24T21:02:07.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50932026-03-24T21:02:07.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50942026-03-24T21:02:07.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50952026-03-24T21:02:07.428ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
50962026-03-24T21:02:07.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50972026-03-24T21:02:07.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50982026-03-24T21:02:07.430ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
50992026-03-24T21:02:07.430ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
51002026-03-24T21:02:07.430ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 194.399051ms iteration = 17 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
51012026-03-24T21:02:08.551ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:59409
51022026-03-24T21:02:08.551ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = c06fa760-2136-4af6-b231-504004f6010c uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
51032026-03-24T21:02:08.551ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = c06fa760-2136-4af6-b231-504004f6010c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
51042026-03-24T21:02:08.551ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = c06fa760-2136-4af6-b231-504004f6010c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
51052026-03-24T21:02:08.554ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 3041 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = c06fa760-2136-4af6-b231-504004f6010c response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
51062026-03-24T21:02:08.554ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = d1e76629-bb77-473e-bfc5-82d94f71bffb uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
51072026-03-24T21:02:08.554ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = d1e76629-bb77-473e-bfc5-82d94f71bffb roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
51082026-03-24T21:02:08.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = d1e76629-bb77-473e-bfc5-82d94f71bffb resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
51092026-03-24T21:02:08.556ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2192 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = d1e76629-bb77-473e-bfc5-82d94f71bffb response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
51102026-03-24T21:02:08.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 379 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 6859d093-9421-4e8d-9a97-8f4dc28dd34c response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
51112026-03-24T21:02:10.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 62 reason = Timeout
51122026-03-24T21:02:10.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51132026-03-24T21:02:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51142026-03-24T21:02:10.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51152026-03-24T21:02:10.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
51162026-03-24T21:02:10.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51172026-03-24T21:02:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51182026-03-24T21:02:10.538ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
51192026-03-24T21:02:10.538ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.387013ms iteration = 62 status = {"config_updated":false}
51202026-03-24T21:02:15.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
51212026-03-24T21:02:15.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 63 reason = Timeout
51222026-03-24T21:02:15.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51232026-03-24T21:02:15.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51242026-03-24T21:02:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51252026-03-24T21:02:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51262026-03-24T21:02:15.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51272026-03-24T21:02:15.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51282026-03-24T21:02:15.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
51292026-03-24T21:02:15.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
51302026-03-24T21:02:15.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51312026-03-24T21:02:15.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51322026-03-24T21:02:15.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51332026-03-24T21:02:15.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51342026-03-24T21:02:15.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51352026-03-24T21:02:15.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
51362026-03-24T21:02:15.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.166943ms iteration = 63 status = {"config_updated":false}
51372026-03-24T21:02:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51382026-03-24T21:02:15.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51392026-03-24T21:02:15.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
51402026-03-24T21:02:15.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
51412026-03-24T21:02:15.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
51422026-03-24T21:02:15.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
51432026-03-24T21:02:15.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51442026-03-24T21:02:15.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51452026-03-24T21:02:15.556ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
51462026-03-24T21:02:15.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
51472026-03-24T21:02:15.561ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
51482026-03-24T21:02:15.561ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.752095ms iteration = 32 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
51492026-03-24T21:02:15.596ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.121996ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
51502026-03-24T21:02:18.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 425 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = f314880b-a74d-4b16-9c3d-773409b3ea7c response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
51512026-03-24T21:02:20.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 30 reason = Timeout
51522026-03-24T21:02:20.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 64 reason = Timeout
51532026-03-24T21:02:20.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
51542026-03-24T21:02:20.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51552026-03-24T21:02:20.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
51562026-03-24T21:02:20.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51572026-03-24T21:02:20.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51582026-03-24T21:02:20.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51592026-03-24T21:02:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
51602026-03-24T21:02:20.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
51612026-03-24T21:02:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
51622026-03-24T21:02:20.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
51632026-03-24T21:02:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
51642026-03-24T21:02:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
51652026-03-24T21:02:20.540ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 9789333b-5bf2-4c58-8e77-63169dbd8708 original_time_started = 2026-03-24 21:02:06.888816 UTC
51662026-03-24T21:02:20.540ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.954154ms iteration = 30 status = {"Loaded":{"collection_id":"9789333b-5bf2-4c58-8e77-63169dbd8708","time_loaded":"2026-03-24T21:02:20.532539545Z","time_started":"2026-03-24T21:02:06.888816Z"}}
51672026-03-24T21:02:20.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
51682026-03-24T21:02:20.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.964334ms iteration = 64 status = {"config_updated":false}
51692026-03-24T21:02:20.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 22 reason = Timeout
51702026-03-24T21:02:20.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 22 reason = Timeout
51712026-03-24T21:02:20.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
51722026-03-24T21:02:20.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
51732026-03-24T21:02:20.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
51742026-03-24T21:02:20.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
51752026-03-24T21:02:20.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
51762026-03-24T21:02:20.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
51772026-03-24T21:02:20.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51782026-03-24T21:02:20.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51792026-03-24T21:02:20.579ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
51802026-03-24T21:02:20.579ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
51812026-03-24T21:02:20.579ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
51822026-03-24T21:02:20.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
51832026-03-24T21:02:20.581ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.099815ms iteration = 22 status = "NoSitrep"
51842026-03-24T21:02:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
51852026-03-24T21:02:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
51862026-03-24T21:02:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
51872026-03-24T21:02:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
51882026-03-24T21:02:20.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51892026-03-24T21:02:20.585ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
51902026-03-24T21:02:20.588ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
51912026-03-24T21:02:20.588ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
51922026-03-24T21:02:20.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.496924ms iteration = 22 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
51932026-03-24T21:02:23.551ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = 39a8bee4-2d07-4958-8d5a-af2faebfb335 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
51942026-03-24T21:02:23.551ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = 39a8bee4-2d07-4958-8d5a-af2faebfb335 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
51952026-03-24T21:02:23.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = 39a8bee4-2d07-4958-8d5a-af2faebfb335 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
51962026-03-24T21:02:23.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 3539 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = 39a8bee4-2d07-4958-8d5a-af2faebfb335 response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
51972026-03-24T21:02:23.555ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = 068f7664-f258-4552-be06-7ea7f2135968 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
51982026-03-24T21:02:23.555ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = 068f7664-f258-4552-be06-7ea7f2135968 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
51992026-03-24T21:02:23.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = 068f7664-f258-4552-be06-7ea7f2135968 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
52002026-03-24T21:02:23.557ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2274 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59409 req_id = 068f7664-f258-4552-be06-7ea7f2135968 response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
52012026-03-24T21:02:25.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
52022026-03-24T21:02:25.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 65 reason = Timeout
52032026-03-24T21:02:25.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52042026-03-24T21:02:25.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
52052026-03-24T21:02:25.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
52062026-03-24T21:02:25.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
52072026-03-24T21:02:25.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52082026-03-24T21:02:25.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52092026-03-24T21:02:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
52102026-03-24T21:02:25.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
52112026-03-24T21:02:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
52122026-03-24T21:02:25.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52132026-03-24T21:02:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
52142026-03-24T21:02:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
52152026-03-24T21:02:25.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
52162026-03-24T21:02:25.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.496572ms iteration = 65 status = {"config_updated":false}
52172026-03-24T21:02:25.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52182026-03-24T21:02:25.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
52192026-03-24T21:02:25.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52202026-03-24T21:02:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
52212026-03-24T21:02:25.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52222026-03-24T21:02:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
52232026-03-24T21:02:25.553ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
52242026-03-24T21:02:25.553ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52252026-03-24T21:02:25.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52262026-03-24T21:02:25.555ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
52272026-03-24T21:02:25.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
52282026-03-24T21:02:25.557ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
52292026-03-24T21:02:25.557ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.668765ms iteration = 33 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
52302026-03-24T21:02:25.557ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.381571ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
52312026-03-24T21:02:28.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 647 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = db0ac770-e682-463b-bf32-71456754a5de response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
52322026-03-24T21:02:30.531ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 66 reason = Timeout
52332026-03-24T21:02:30.531ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
52342026-03-24T21:02:30.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
52352026-03-24T21:02:30.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52362026-03-24T21:02:30.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
52372026-03-24T21:02:30.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
52382026-03-24T21:02:30.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
52392026-03-24T21:02:30.538ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
52402026-03-24T21:02:30.538ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.266869ms iteration = 66 status = {"config_updated":false}
52412026-03-24T21:02:35.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
52422026-03-24T21:02:35.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 12 reason = Timeout
52432026-03-24T21:02:35.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
52442026-03-24T21:02:35.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
52452026-03-24T21:02:35.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
52462026-03-24T21:02:35.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
52472026-03-24T21:02:35.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
52482026-03-24T21:02:35.522ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 2.099677ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
52492026-03-24T21:02:35.523ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1141, tv_nsec: 737284636 } })
52502026-03-24T21:02:35.523ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1141, tv_nsec: 737346658 } })
52512026-03-24T21:02:35.523ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
52522026-03-24T21:02:35.523ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1141, tv_nsec: 738115068 } })
52532026-03-24T21:02:35.525ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
52542026-03-24T21:02:35.525ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
52552026-03-24T21:02:35.525ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
52562026-03-24T21:02:35.525ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 4.859442ms iteration = 12 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
52572026-03-24T21:02:35.525ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "99aee8ce-1215-4990-a869-d904cb81d406", "content-length": "26", "date": "Tue, 24 Mar 2026 21:02:35 GMT"} })
52582026-03-24T21:02:35.525ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
52592026-03-24T21:02:35.525ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
52602026-03-24T21:02:35.525ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1141, tv_nsec: 740127253 } })
52612026-03-24T21:02:35.527ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 7.252986ms iteration = 12 status = {}
52622026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
52632026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
52642026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52652026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
52662026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52672026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
52682026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 31 reason = Timeout
52692026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 67 reason = Timeout
52702026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
52712026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
52722026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
52732026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
52742026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
52752026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
52762026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 18 reason = Timeout
52772026-03-24T21:02:35.532ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52782026-03-24T21:02:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
52792026-03-24T21:02:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
52802026-03-24T21:02:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52812026-03-24T21:02:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52822026-03-24T21:02:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
52832026-03-24T21:02:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
52842026-03-24T21:02:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52852026-03-24T21:02:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
52862026-03-24T21:02:35.535ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52872026-03-24T21:02:35.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52882026-03-24T21:02:35.535ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
52892026-03-24T21:02:35.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52902026-03-24T21:02:35.536ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52912026-03-24T21:02:35.536ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52922026-03-24T21:02:35.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
52932026-03-24T21:02:35.539ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52942026-03-24T21:02:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
52952026-03-24T21:02:35.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
52962026-03-24T21:02:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
52972026-03-24T21:02:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
52982026-03-24T21:02:35.540ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52992026-03-24T21:02:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
53002026-03-24T21:02:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
53012026-03-24T21:02:35.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53022026-03-24T21:02:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
53032026-03-24T21:02:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
53042026-03-24T21:02:35.542ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
53052026-03-24T21:02:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
53062026-03-24T21:02:35.542ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
53072026-03-24T21:02:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
53082026-03-24T21:02:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
53092026-03-24T21:02:35.542ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53102026-03-24T21:02:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
53112026-03-24T21:02:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53122026-03-24T21:02:35.543ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
53132026-03-24T21:02:35.543ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1141, tv_nsec: 758211670 } })
53142026-03-24T21:02:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
53152026-03-24T21:02:35.544ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
53162026-03-24T21:02:35.544ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 9789333b-5bf2-4c58-8e77-63169dbd8708 original_time_started = 2026-03-24 21:02:06.888816 UTC
53172026-03-24T21:02:35.544ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 12.008204ms iteration = 31 status = {"Loaded":{"collection_id":"9789333b-5bf2-4c58-8e77-63169dbd8708","time_loaded":"2026-03-24T21:02:35.532704268Z","time_started":"2026-03-24T21:02:06.888816Z"}}
53182026-03-24T21:02:35.544ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1141, tv_nsec: 759043513 } })
53192026-03-24T21:02:35.544ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
53202026-03-24T21:02:35.544ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 12.081855ms iteration = 67 status = {"config_updated":false}
53212026-03-24T21:02:35.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53222026-03-24T21:02:35.545ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53232026-03-24T21:02:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
53242026-03-24T21:02:35.546ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53252026-03-24T21:02:35.546ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
53262026-03-24T21:02:35.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
53272026-03-24T21:02:35.546ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53282026-03-24T21:02:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53292026-03-24T21:02:35.546ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
53302026-03-24T21:02:35.546ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
53312026-03-24T21:02:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
53322026-03-24T21:02:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
53332026-03-24T21:02:35.546ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5d50791-ccdd-4fdd-9083-26132fc10db6", "content-length": "26", "date": "Tue, 24 Mar 2026 21:02:35 GMT"} })
53342026-03-24T21:02:35.546ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
53352026-03-24T21:02:35.546ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
53362026-03-24T21:02:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
53372026-03-24T21:02:35.548ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
53382026-03-24T21:02:35.548ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53392026-03-24T21:02:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
53402026-03-24T21:02:35.548ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1141, tv_nsec: 763011620 } })
53412026-03-24T21:02:35.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53422026-03-24T21:02:35.551ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1141, tv_nsec: 765553208 } })
53432026-03-24T21:02:35.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
53442026-03-24T21:02:35.551ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53452026-03-24T21:02:35.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53462026-03-24T21:02:35.552ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53472026-03-24T21:02:35.552ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
53482026-03-24T21:02:35.552ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 19.816854ms iteration = 18 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
53492026-03-24T21:02:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53502026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
53512026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
53522026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
53532026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
53542026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
53552026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
53562026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
53572026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
53582026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
53592026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
53602026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
53612026-03-24T21:02:35.554ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
53622026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 12 reason = Timeout
53632026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53642026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
53652026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
53662026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
53672026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
53682026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
53692026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
53702026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
53712026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
53722026-03-24T21:02:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53732026-03-24T21:02:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
53742026-03-24T21:02:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
53752026-03-24T21:02:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
53762026-03-24T21:02:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
53772026-03-24T21:02:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
53782026-03-24T21:02:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
53792026-03-24T21:02:35.555ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
53802026-03-24T21:02:35.555ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
53812026-03-24T21:02:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
53822026-03-24T21:02:35.556ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53832026-03-24T21:02:35.557ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53842026-03-24T21:02:35.557ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 12 reason = Timeout
53852026-03-24T21:02:35.557ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
53862026-03-24T21:02:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
53872026-03-24T21:02:35.558ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53882026-03-24T21:02:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
53892026-03-24T21:02:35.559ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
53902026-03-24T21:02:35.559ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
53912026-03-24T21:02:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
53922026-03-24T21:02:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
53932026-03-24T21:02:35.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
53942026-03-24T21:02:35.561ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53952026-03-24T21:02:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
53962026-03-24T21:02:35.564ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
53972026-03-24T21:02:35.564ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
53982026-03-24T21:02:35.564ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53992026-03-24T21:02:35.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
54002026-03-24T21:02:35.566ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54012026-03-24T21:02:35.567ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 9.800115ms iteration = 12 status = {"errors":[],"requests_created_ok":[]}
54022026-03-24T21:02:35.568ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
54032026-03-24T21:02:35.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54042026-03-24T21:02:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54052026-03-24T21:02:35.569ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
54062026-03-24T21:02:35.569ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
54072026-03-24T21:02:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
54082026-03-24T21:02:35.573ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
54092026-03-24T21:02:35.573ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
54102026-03-24T21:02:35.573ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
54112026-03-24T21:02:35.573ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
54122026-03-24T21:02:35.573ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
54132026-03-24T21:02:35.573ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
54142026-03-24T21:02:35.573ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
54152026-03-24T21:02:35.573ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
54162026-03-24T21:02:35.573ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
54172026-03-24T21:02:35.573ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
54182026-03-24T21:02:35.574ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6c25a24-ea84-4193-81a1-25cfeab8787e", "content-length": "372", "date": "Tue, 24 Mar 2026 21:02:35 GMT"} })
54192026-03-24T21:02:35.574ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54202026-03-24T21:02:35.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
54212026-03-24T21:02:35.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54222026-03-24T21:02:35.575ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 12 reason = Timeout
54232026-03-24T21:02:35.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 23 reason = Timeout
54242026-03-24T21:02:35.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 23 reason = Timeout
54252026-03-24T21:02:35.575ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.561\u{b5}s iteration = 12 status = {"disabled":true,"errors":[],"sps":[]}
54262026-03-24T21:02:35.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
54272026-03-24T21:02:35.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
54282026-03-24T21:02:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
54292026-03-24T21:02:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
54302026-03-24T21:02:35.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
54312026-03-24T21:02:35.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
54322026-03-24T21:02:35.577ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
54332026-03-24T21:02:35.577ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
54342026-03-24T21:02:35.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54352026-03-24T21:02:35.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54362026-03-24T21:02:35.580ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
54372026-03-24T21:02:35.581ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
54382026-03-24T21:02:35.581ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
54392026-03-24T21:02:35.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
54402026-03-24T21:02:35.582ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
54412026-03-24T21:02:35.582ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.272176ms iteration = 34 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
54422026-03-24T21:02:35.583ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 8.237862ms iteration = 23 status = "NoSitrep"
54432026-03-24T21:02:35.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
54442026-03-24T21:02:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
54452026-03-24T21:02:35.585ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
54462026-03-24T21:02:35.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
54472026-03-24T21:02:35.592ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54482026-03-24T21:02:35.594ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
54492026-03-24T21:02:35.604ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
54502026-03-24T21:02:35.604ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
54512026-03-24T21:02:35.604ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 28.768326ms iteration = 23 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
54522026-03-24T21:02:35.618ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54532026-03-24T21:02:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
54542026-03-24T21:02:35.623ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
54552026-03-24T21:02:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
54562026-03-24T21:02:35.628ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
54572026-03-24T21:02:35.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
54582026-03-24T21:02:35.628ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 96.067741ms iteration = 12 status = 0
54592026-03-24T21:02:35.631ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 85.044943ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
54602026-03-24T21:02:35.631ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
54612026-03-24T21:02:35.631ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 77.223582ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
54622026-03-24T21:02:35.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
54632026-03-24T21:02:35.632ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 78.24513ms iteration = 12 status = {"errors":[],"finish_invoked_ok":[]}
54642026-03-24T21:02:35.633ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
54652026-03-24T21:02:35.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
54662026-03-24T21:02:35.633ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 78.938329ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
54672026-03-24T21:02:35.634ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 80.259094ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
54682026-03-24T21:02:35.635ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 102.794592ms iteration = 21 status = {"physical_disks_added":0}
54692026-03-24T21:02:35.637ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
54702026-03-24T21:02:35.637ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
54712026-03-24T21:02:35.637ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 82.70594ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
54722026-03-24T21:02:35.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54732026-03-24T21:02:35.637ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
54742026-03-24T21:02:35.637ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
54752026-03-24T21:02:35.637ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
54762026-03-24T21:02:35.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
54772026-03-24T21:02:35.652ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
54782026-03-24T21:02:35.652ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
54792026-03-24T21:02:35.654ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "396c7d07-ed38-4399-b1ad-ecb99720e8df", "date": "Tue, 24 Mar 2026 21:02:35 GMT"} })
54802026-03-24T21:02:35.654ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54812026-03-24T21:02:35.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54822026-03-24T21:02:35.657ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54832026-03-24T21:02:35.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54842026-03-24T21:02:35.660ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
54852026-03-24T21:02:35.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
54862026-03-24T21:02:35.663ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
54872026-03-24T21:02:35.663ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
54882026-03-24T21:02:35.664ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8f31e512-bb8c-41a0-ae7e-68cf44f19ab0", "date": "Tue, 24 Mar 2026 21:02:35 GMT"} })
54892026-03-24T21:02:35.664ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 109.491953ms iteration = 18 status = {}
54902026-03-24T21:02:35.681ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
54912026-03-24T21:02:35.681ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
54922026-03-24T21:02:35.681ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
54932026-03-24T21:02:35.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
54942026-03-24T21:02:35.682ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
54952026-03-24T21:02:35.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
54962026-03-24T21:02:35.684ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 129.382099ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
54972026-03-24T21:02:35.688ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
54982026-03-24T21:02:35.690ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "263cc59b-5776-4c01-8629-49425ae8fd45", "content-length": "691", "date": "Tue, 24 Mar 2026 21:02:35 GMT"} })
54992026-03-24T21:02:35.690ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
55002026-03-24T21:02:35.690ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
55012026-03-24T21:02:35.690ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
55022026-03-24T21:02:35.690ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 158.131204ms iteration = 12 status = {}
55032026-03-24T21:02:35.734ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
55042026-03-24T21:02:35.734ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
55052026-03-24T21:02:35.734ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 179.530031ms iteration = 12 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
55062026-03-24T21:02:35.865ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6f62d2fa-4df5-4bea-abc8-188f8838de89", "date": "Tue, 24 Mar 2026 21:02:35 GMT"} })
55072026-03-24T21:02:35.865ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
55082026-03-24T21:02:35.865ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55092026-03-24T21:02:35.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55102026-03-24T21:02:35.868ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55112026-03-24T21:02:35.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55122026-03-24T21:02:35.871ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55132026-03-24T21:02:35.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55142026-03-24T21:02:35.876ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
55152026-03-24T21:02:35.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
55162026-03-24T21:02:35.878ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 346.275308ms iteration = 12 status = {}
55172026-03-24T21:02:38.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 427 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 1666ae9c-43e4-433c-941b-3cf8f6348463 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
55182026-03-24T21:02:40.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 68 reason = Timeout
55192026-03-24T21:02:40.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
55202026-03-24T21:02:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
55212026-03-24T21:02:40.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55222026-03-24T21:02:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
55232026-03-24T21:02:40.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
55242026-03-24T21:02:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
55252026-03-24T21:02:40.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
55262026-03-24T21:02:40.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.469831ms iteration = 68 status = {"config_updated":false}
55272026-03-24T21:02:45.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
55282026-03-24T21:02:45.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 69 reason = Timeout
55292026-03-24T21:02:45.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55302026-03-24T21:02:45.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
55312026-03-24T21:02:45.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
55322026-03-24T21:02:45.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
55332026-03-24T21:02:45.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55342026-03-24T21:02:45.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55352026-03-24T21:02:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
55362026-03-24T21:02:45.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
55372026-03-24T21:02:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
55382026-03-24T21:02:45.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55392026-03-24T21:02:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
55402026-03-24T21:02:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
55412026-03-24T21:02:45.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
55422026-03-24T21:02:45.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.902936ms iteration = 69 status = {"config_updated":false}
55432026-03-24T21:02:45.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55442026-03-24T21:02:45.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
55452026-03-24T21:02:45.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55462026-03-24T21:02:45.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
55472026-03-24T21:02:45.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55482026-03-24T21:02:45.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
55492026-03-24T21:02:45.553ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
55502026-03-24T21:02:45.553ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55512026-03-24T21:02:45.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55522026-03-24T21:02:45.555ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55532026-03-24T21:02:45.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55542026-03-24T21:02:45.557ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.315709ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
55552026-03-24T21:02:45.675ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
55562026-03-24T21:02:45.675ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 142.942015ms iteration = 35 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
55572026-03-24T21:02:48.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 424 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 506f69a7-470f-4de0-a196-4547843da45f response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
55582026-03-24T21:02:50.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 32 reason = Timeout
55592026-03-24T21:02:50.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 70 reason = Timeout
55602026-03-24T21:02:50.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
55612026-03-24T21:02:50.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
55622026-03-24T21:02:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
55632026-03-24T21:02:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
55642026-03-24T21:02:50.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55652026-03-24T21:02:50.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55662026-03-24T21:02:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
55672026-03-24T21:02:50.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
55682026-03-24T21:02:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
55692026-03-24T21:02:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
55702026-03-24T21:02:50.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
55712026-03-24T21:02:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
55722026-03-24T21:02:50.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
55732026-03-24T21:02:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.822281ms iteration = 70 status = {"config_updated":false}
55742026-03-24T21:02:50.540ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 9789333b-5bf2-4c58-8e77-63169dbd8708 original_time_started = 2026-03-24 21:02:06.888816 UTC
55752026-03-24T21:02:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.401727ms iteration = 32 status = {"Loaded":{"collection_id":"9789333b-5bf2-4c58-8e77-63169dbd8708","time_loaded":"2026-03-24T21:02:50.532425832Z","time_started":"2026-03-24T21:02:06.888816Z"}}
55762026-03-24T21:02:50.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 24 reason = Timeout
55772026-03-24T21:02:50.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
55782026-03-24T21:02:50.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 24 reason = Timeout
55792026-03-24T21:02:50.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
55802026-03-24T21:02:50.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
55812026-03-24T21:02:50.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
55822026-03-24T21:02:50.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
55832026-03-24T21:02:50.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
55842026-03-24T21:02:50.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55852026-03-24T21:02:50.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55862026-03-24T21:02:50.578ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
55872026-03-24T21:02:50.579ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
55882026-03-24T21:02:50.579ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
55892026-03-24T21:02:50.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
55902026-03-24T21:02:50.580ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
55912026-03-24T21:02:50.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
55922026-03-24T21:02:50.581ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.876668ms iteration = 24 status = "NoSitrep"
55932026-03-24T21:02:50.581ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
55942026-03-24T21:02:50.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
55952026-03-24T21:02:50.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55962026-03-24T21:02:50.584ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
55972026-03-24T21:02:50.586ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
55982026-03-24T21:02:50.586ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
55992026-03-24T21:02:50.586ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.502481ms iteration = 24 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
56002026-03-24T21:02:55.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
56012026-03-24T21:02:55.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 71 reason = Timeout
56022026-03-24T21:02:55.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56032026-03-24T21:02:55.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56042026-03-24T21:02:55.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56052026-03-24T21:02:55.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56062026-03-24T21:02:55.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56072026-03-24T21:02:55.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56082026-03-24T21:02:55.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
56092026-03-24T21:02:55.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56102026-03-24T21:02:55.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
56112026-03-24T21:02:55.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56122026-03-24T21:02:55.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56132026-03-24T21:02:55.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56142026-03-24T21:02:55.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
56152026-03-24T21:02:55.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.889166ms iteration = 71 status = {"config_updated":false}
56162026-03-24T21:02:55.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56172026-03-24T21:02:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56182026-03-24T21:02:55.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56192026-03-24T21:02:55.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
56202026-03-24T21:02:55.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56212026-03-24T21:02:55.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56222026-03-24T21:02:55.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
56232026-03-24T21:02:55.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56242026-03-24T21:02:55.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56252026-03-24T21:02:55.555ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
56262026-03-24T21:02:55.556ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.896154ms iteration = 36 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
56272026-03-24T21:02:55.556ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56282026-03-24T21:02:55.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56292026-03-24T21:02:55.557ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.175556ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
56302026-03-24T21:02:58.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 371 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 0c5e93c0-9a64-43ba-90af-f0e1e0042052 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
56312026-03-24T21:03:00.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 72 reason = Timeout
56322026-03-24T21:03:00.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56332026-03-24T21:03:00.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56342026-03-24T21:03:00.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56352026-03-24T21:03:00.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
56362026-03-24T21:03:00.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56372026-03-24T21:03:00.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56382026-03-24T21:03:00.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
56392026-03-24T21:03:00.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.901276ms iteration = 72 status = {"config_updated":false}
56402026-03-24T21:03:05.515ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
56412026-03-24T21:03:05.515ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
56422026-03-24T21:03:05.515ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
56432026-03-24T21:03:05.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
56442026-03-24T21:03:05.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
56452026-03-24T21:03:05.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
56462026-03-24T21:03:05.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56472026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
56482026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
56492026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
56502026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 13 reason = Timeout
56512026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
56522026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
56532026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
56542026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
56552026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
56562026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
56572026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
56582026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
56592026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
56602026-03-24T21:03:05.520ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2026-03-24 20:53:05.520898644 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
56612026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
56622026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
56632026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
56642026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
56652026-03-24T21:03:05.520ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
56662026-03-24T21:03:05.520ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
56672026-03-24T21:03:05.520ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = ExternalNetworking step id = Ensure
56682026-03-24T21:03:05.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
56692026-03-24T21:03:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
56702026-03-24T21:03:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
56712026-03-24T21:03:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
56722026-03-24T21:03:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
56732026-03-24T21:03:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
56742026-03-24T21:03:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
56752026-03-24T21:03:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("93b6e989-1f7d-4884-aec9-ecbc076ed6c4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 735928900 } })
56762026-03-24T21:03:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("27f3a9c5-67c7-41c5-994d-9db513cda557.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2f7ea2ed-fd45-4cd7-9fb1-f48119541118.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 735958710 } })
56772026-03-24T21:03:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
56782026-03-24T21:03:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.352677ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
56792026-03-24T21:03:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
56802026-03-24T21:03:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
56812026-03-24T21:03:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.227713ms iteration = 7 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
56822026-03-24T21:03:05.522ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
56832026-03-24T21:03:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
56842026-03-24T21:03:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
56852026-03-24T21:03:05.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
56862026-03-24T21:03:05.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
56872026-03-24T21:03:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "c6bc978f-bf32-4163-92c4-b2482fdb47a5", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
56882026-03-24T21:03:05.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
56892026-03-24T21:03:05.525ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
56902026-03-24T21:03:05.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
56912026-03-24T21:03:05.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "db95f297-7e85-441c-b848-3031b25edea2", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56922026-03-24T21:03:05.525ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56932026-03-24T21:03:05.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "fc55b48a-0a7e-4005-9050-c02c66f7036f", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56942026-03-24T21:03:05.525ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56952026-03-24T21:03:05.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "33776012-6265-4c49-8bbc-34ffdf5f79e3", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56962026-03-24T21:03:05.525ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56972026-03-24T21:03:05.525ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
56982026-03-24T21:03:05.525ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 5.058637ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
56992026-03-24T21:03:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "cdf39715-bc96-4549-8750-49432281d5a4", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
57002026-03-24T21:03:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
57012026-03-24T21:03:05.526ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
57022026-03-24T21:03:05.526ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 5.422666ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
57032026-03-24T21:03:05.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57042026-03-24T21:03:05.526ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2026-03-24 20:53:05.520898644 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
57052026-03-24T21:03:05.526ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.760115ms iteration = 7 status = {"expiration":"2026-03-24T20:53:05.520898644Z","pruned":[]}
57062026-03-24T21:03:05.527ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 741734006 } })
57072026-03-24T21:03:05.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57082026-03-24T21:03:05.527ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
57092026-03-24T21:03:05.527ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
57102026-03-24T21:03:05.527ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
57112026-03-24T21:03:05.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
57122026-03-24T21:03:05.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
57132026-03-24T21:03:05.530ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57142026-03-24T21:03:05.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
57152026-03-24T21:03:05.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
57162026-03-24T21:03:05.530ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57172026-03-24T21:03:05.531ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
57182026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 10.342059ms iteration = 13 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
57192026-03-24T21:03:05.531ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 745945000 } })
57202026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
57212026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Timeout
57222026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
57232026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
57242026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 33 reason = Timeout
57252026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
57262026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 18 reason = Timeout
57272026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
57282026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
57292026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57302026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
57312026-03-24T21:03:05.531ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57322026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 73 reason = Timeout
57332026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
57342026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
57352026-03-24T21:03:05.532ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
57362026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
57372026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
57382026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57392026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
57402026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
57412026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57422026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
57432026-03-24T21:03:05.532ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
57442026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
57452026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
57462026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
57472026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
57482026-03-24T21:03:05.532ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 747025129 } })
57492026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
57502026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57512026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
57522026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
57532026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
57542026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57552026-03-24T21:03:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
57562026-03-24T21:03:05.533ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
57572026-03-24T21:03:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
57582026-03-24T21:03:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57592026-03-24T21:03:05.533ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
57602026-03-24T21:03:05.534ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57612026-03-24T21:03:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
57622026-03-24T21:03:05.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
57632026-03-24T21:03:05.534ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
57642026-03-24T21:03:05.534ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
57652026-03-24T21:03:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
57662026-03-24T21:03:05.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
57672026-03-24T21:03:05.534ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f4ac75c-cc55-4fab-bccc-8faa04cb5e1c", "content-length": "26", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} })
57682026-03-24T21:03:05.534ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
57692026-03-24T21:03:05.534ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
57702026-03-24T21:03:05.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
57712026-03-24T21:03:05.535ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
57722026-03-24T21:03:05.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57732026-03-24T21:03:05.535ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
57742026-03-24T21:03:05.535ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 749876936 } })
57752026-03-24T21:03:05.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57762026-03-24T21:03:05.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
57772026-03-24T21:03:05.536ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57782026-03-24T21:03:05.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57792026-03-24T21:03:05.536ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57802026-03-24T21:03:05.536ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57812026-03-24T21:03:05.537ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 16.727721ms iteration = 13 status = {}
57822026-03-24T21:03:05.538ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
57832026-03-24T21:03:05.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
57842026-03-24T21:03:05.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
57852026-03-24T21:03:05.539ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 18.618852ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
57862026-03-24T21:03:05.539ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57872026-03-24T21:03:05.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57882026-03-24T21:03:05.540ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
57892026-03-24T21:03:05.540ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
57902026-03-24T21:03:05.540ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
57912026-03-24T21:03:05.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
57922026-03-24T21:03:05.540ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57932026-03-24T21:03:05.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57942026-03-24T21:03:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
57952026-03-24T21:03:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57962026-03-24T21:03:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
57972026-03-24T21:03:05.542ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57982026-03-24T21:03:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
57992026-03-24T21:03:05.542ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
58002026-03-24T21:03:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
58012026-03-24T21:03:05.542ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
58022026-03-24T21:03:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
58032026-03-24T21:03:05.542ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58042026-03-24T21:03:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
58052026-03-24T21:03:05.543ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
58062026-03-24T21:03:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
58072026-03-24T21:03:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
58082026-03-24T21:03:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
58092026-03-24T21:03:05.543ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
58102026-03-24T21:03:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58112026-03-24T21:03:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
58122026-03-24T21:03:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
58132026-03-24T21:03:05.544ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
58142026-03-24T21:03:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
58152026-03-24T21:03:05.544ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58162026-03-24T21:03:05.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58172026-03-24T21:03:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
58182026-03-24T21:03:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
58192026-03-24T21:03:05.544ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
58202026-03-24T21:03:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
58212026-03-24T21:03:05.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
58222026-03-24T21:03:05.546ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
58232026-03-24T21:03:05.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
58242026-03-24T21:03:05.546ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58252026-03-24T21:03:05.546ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
58262026-03-24T21:03:05.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
58272026-03-24T21:03:05.546ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
58282026-03-24T21:03:05.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
58292026-03-24T21:03:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
58302026-03-24T21:03:05.547ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
58312026-03-24T21:03:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
58322026-03-24T21:03:05.547ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 15.348364ms iteration = 73 status = {"config_updated":false}
58332026-03-24T21:03:05.548ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
58342026-03-24T21:03:05.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
58352026-03-24T21:03:05.549ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
58362026-03-24T21:03:05.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
58372026-03-24T21:03:05.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58382026-03-24T21:03:05.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
58392026-03-24T21:03:05.550ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58402026-03-24T21:03:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
58412026-03-24T21:03:05.551ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
58422026-03-24T21:03:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58432026-03-24T21:03:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
58442026-03-24T21:03:05.551ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
58452026-03-24T21:03:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
58462026-03-24T21:03:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
58472026-03-24T21:03:05.551ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58482026-03-24T21:03:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58492026-03-24T21:03:05.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58502026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58512026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
58522026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
58532026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
58542026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
58552026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
58562026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
58572026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
58582026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
58592026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
58602026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
58612026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
58622026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
58632026-03-24T21:03:05.553ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58642026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
58652026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
58662026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
58672026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
58682026-03-24T21:03:05.554ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
58692026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
58702026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
58712026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58722026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
58732026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
58742026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
58752026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
58762026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
58772026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
58782026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
58792026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
58802026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58812026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
58822026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
58832026-03-24T21:03:05.554ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 768867718 } })
58842026-03-24T21:03:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
58852026-03-24T21:03:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58862026-03-24T21:03:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
58872026-03-24T21:03:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
58882026-03-24T21:03:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
58892026-03-24T21:03:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
58902026-03-24T21:03:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
58912026-03-24T21:03:05.555ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
58922026-03-24T21:03:05.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58932026-03-24T21:03:05.555ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
58942026-03-24T21:03:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
58952026-03-24T21:03:05.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58962026-03-24T21:03:05.556ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 770649656 } })
58972026-03-24T21:03:05.556ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58982026-03-24T21:03:05.556ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 7 reason = Timeout
58992026-03-24T21:03:05.556ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 16 reason = Timeout
59002026-03-24T21:03:05.556ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
59012026-03-24T21:03:05.556ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
59022026-03-24T21:03:05.556ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 14.611\u{b5}s iteration = 16 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
59032026-03-24T21:03:05.556ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 7 reason = Timeout
59042026-03-24T21:03:05.556ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 13 reason = Timeout
59052026-03-24T21:03:05.556ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
59062026-03-24T21:03:05.556ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
59072026-03-24T21:03:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
59082026-03-24T21:03:05.557ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59092026-03-24T21:03:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
59102026-03-24T21:03:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
59112026-03-24T21:03:05.557ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
59122026-03-24T21:03:05.558ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
59132026-03-24T21:03:05.558ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "af0bdf7c-55a6-46e6-82c9-f475eecdd4da", "content-length": "26", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} })
59142026-03-24T21:03:05.558ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
59152026-03-24T21:03:05.558ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
59162026-03-24T21:03:05.559ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59172026-03-24T21:03:05.559ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
59182026-03-24T21:03:05.559ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 9789333b-5bf2-4c58-8e77-63169dbd8708 original_time_started = 2026-03-24 21:02:06.888816 UTC
59192026-03-24T21:03:05.559ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 27.755689ms iteration = 33 status = {"Loaded":{"collection_id":"9789333b-5bf2-4c58-8e77-63169dbd8708","time_loaded":"2026-03-24T21:03:05.531899340Z","time_started":"2026-03-24T21:02:06.888816Z"}}
59202026-03-24T21:03:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
59212026-03-24T21:03:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59222026-03-24T21:03:05.561ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 775025584 } })
59232026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
59242026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
59252026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
59262026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
59272026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
59282026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
59292026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59302026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
59312026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
59322026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
59332026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
59342026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
59352026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
59362026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
59372026-03-24T21:03:05.561ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59382026-03-24T21:03:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
59392026-03-24T21:03:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
59402026-03-24T21:03:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
59412026-03-24T21:03:05.563ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 777935462 } })
59422026-03-24T21:03:05.567ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
59432026-03-24T21:03:05.567ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59442026-03-24T21:03:05.567ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
59452026-03-24T21:03:05.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
59462026-03-24T21:03:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59472026-03-24T21:03:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
59482026-03-24T21:03:05.567ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59492026-03-24T21:03:05.567ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
59502026-03-24T21:03:05.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59512026-03-24T21:03:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59522026-03-24T21:03:05.568ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 14.051169ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
59532026-03-24T21:03:05.568ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 36.378171ms iteration = 13 status = 0
59542026-03-24T21:03:05.568ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
59552026-03-24T21:03:05.568ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
59562026-03-24T21:03:05.568ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 14.551992ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
59572026-03-24T21:03:05.571ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59582026-03-24T21:03:05.571ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
59592026-03-24T21:03:05.571ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
59602026-03-24T21:03:05.571ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59612026-03-24T21:03:05.571ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59622026-03-24T21:03:05.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
59632026-03-24T21:03:05.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59642026-03-24T21:03:05.573ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59652026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 7 reason = Timeout
59662026-03-24T21:03:05.575ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
59672026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
59682026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 13 reason = Timeout
59692026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.36\u{b5}s iteration = 13 status = {"disabled":true,"errors":[],"sps":[]}
59702026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): activating background_task = trust_quorum_manager iteration = 7 reason = Timeout
59712026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
59722026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): activating background_task = attached_subnet_manager iteration = 7 reason = Timeout
59732026-03-24T21:03:05.575ZINFOnexus (BackgroundTasks): starting attached subnet manager work background_task = attached_subnet_manager file = nexus/src/app/background/tasks/attached_subnets.rs:374
59742026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 25 reason = Timeout
59752026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
59762026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 25 reason = Timeout
59772026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
59782026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
59792026-03-24T21:03:05.575ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
59802026-03-24T21:03:05.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
59812026-03-24T21:03:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
59822026-03-24T21:03:05.576ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1171, tv_nsec: 790510611 } })
59832026-03-24T21:03:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
59842026-03-24T21:03:05.576ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59852026-03-24T21:03:05.577ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59862026-03-24T21:03:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
59872026-03-24T21:03:05.577ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59882026-03-24T21:03:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59892026-03-24T21:03:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59902026-03-24T21:03:05.577ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
59912026-03-24T21:03:05.577ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 20.882133ms iteration = 7 status = {"by_rx":{},"error":null}
59922026-03-24T21:03:05.578ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
59932026-03-24T21:03:05.581ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
59942026-03-24T21:03:05.581ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
59952026-03-24T21:03:05.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
59962026-03-24T21:03:05.582ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
59972026-03-24T21:03:05.583ZDEBGnexus (BackgroundTasks): roles background_task = attached_subnet_manager roles = RoleSet { roles: {} }
59982026-03-24T21:03:05.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = attached_subnet_manager resource = Database result = Ok(())
59992026-03-24T21:03:05.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
60002026-03-24T21:03:05.583ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
60012026-03-24T21:03:05.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
60022026-03-24T21:03:05.584ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
60032026-03-24T21:03:05.584ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 30.956775ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
60042026-03-24T21:03:05.587ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60052026-03-24T21:03:05.591ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60062026-03-24T21:03:05.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
60072026-03-24T21:03:05.591ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
60082026-03-24T21:03:05.591ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
60092026-03-24T21:03:05.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
60102026-03-24T21:03:05.593ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
60112026-03-24T21:03:05.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
60122026-03-24T21:03:05.593ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
60132026-03-24T21:03:05.593ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60142026-03-24T21:03:05.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
60152026-03-24T21:03:05.594ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
60162026-03-24T21:03:05.594ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
60172026-03-24T21:03:05.594ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60182026-03-24T21:03:05.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60192026-03-24T21:03:05.595ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60202026-03-24T21:03:05.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60212026-03-24T21:03:05.596ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
60222026-03-24T21:03:05.597ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60232026-03-24T21:03:05.597ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
60242026-03-24T21:03:05.597ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
60252026-03-24T21:03:05.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
60262026-03-24T21:03:05.598ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
60272026-03-24T21:03:05.599ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 45.490156ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
60282026-03-24T21:03:05.600ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
60292026-03-24T21:03:05.600ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
60302026-03-24T21:03:05.600ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2026-03-24 20:57:05.047625 UTC dns_group = internal generation = 1
60312026-03-24T21:03:05.600ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 85.774483ms iteration = 7 status = {"generation":1}
60322026-03-24T21:03:05.600ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
60332026-03-24T21:03:05.601ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
60342026-03-24T21:03:05.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
60352026-03-24T21:03:05.601ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
60362026-03-24T21:03:05.601ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
60372026-03-24T21:03:05.601ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
60382026-03-24T21:03:05.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
60392026-03-24T21:03:05.601ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Fleet result = Ok(())
60402026-03-24T21:03:05.601ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
60412026-03-24T21:03:05.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
60422026-03-24T21:03:05.603ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
60432026-03-24T21:03:05.603ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
60442026-03-24T21:03:05.603ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2026-03-24 20:57:07.125641 UTC dns_group = external generation = 2
60452026-03-24T21:03:05.603ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 82.695739ms iteration = 7 status = {"generation":2}
60462026-03-24T21:03:05.604ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
60472026-03-24T21:03:05.613ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
60482026-03-24T21:03:05.614ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = SledList step id = Fetch
60492026-03-24T21:03:05.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60502026-03-24T21:03:05.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60512026-03-24T21:03:05.622ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60522026-03-24T21:03:05.625ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 68.436696ms iteration = 7 status = {"dispatched":[],"errors":[],"glob_version":"242.0.0","globs_reprocessed":{},"no_receivers":[]}
60532026-03-24T21:03:05.625ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
60542026-03-24T21:03:05.627ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
60552026-03-24T21:03:05.627ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
60562026-03-24T21:03:05.627ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
60572026-03-24T21:03:05.627ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
60582026-03-24T21:03:05.627ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
60592026-03-24T21:03:05.627ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
60602026-03-24T21:03:05.627ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
60612026-03-24T21:03:05.627ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60622026-03-24T21:03:05.628ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60632026-03-24T21:03:05.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60642026-03-24T21:03:05.631ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
60652026-03-24T21:03:05.631ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
60662026-03-24T21:03:05.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
60672026-03-24T21:03:05.634ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
60682026-03-24T21:03:05.634ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
60692026-03-24T21:03:05.637ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d703c6d6-088d-49bc-82e7-0a58ea9d638e", "content-length": "372", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} })
60702026-03-24T21:03:05.637ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60712026-03-24T21:03:05.637ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
60722026-03-24T21:03:05.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60732026-03-24T21:03:05.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
60742026-03-24T21:03:05.639ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 63.612475ms iteration = 25 status = "NoSitrep"
60752026-03-24T21:03:05.639ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
60762026-03-24T21:03:05.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
60772026-03-24T21:03:05.641ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60782026-03-24T21:03:05.644ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
60792026-03-24T21:03:05.644ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
60802026-03-24T21:03:05.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
60812026-03-24T21:03:05.645ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60822026-03-24T21:03:05.647ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
60832026-03-24T21:03:05.650ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
60842026-03-24T21:03:05.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
60852026-03-24T21:03:05.651ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
60862026-03-24T21:03:05.651ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 119.682898ms iteration = 37 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
60872026-03-24T21:03:05.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60882026-03-24T21:03:05.655ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
60892026-03-24T21:03:05.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
60902026-03-24T21:03:05.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
60912026-03-24T21:03:05.660ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 128.116555ms iteration = 4 status = {"nsuccess":5}
60922026-03-24T21:03:05.660ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
60932026-03-24T21:03:05.660ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
60942026-03-24T21:03:05.660ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 85.228578ms iteration = 25 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
60952026-03-24T21:03:05.662ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60962026-03-24T21:03:05.662ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
60972026-03-24T21:03:05.663ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
60982026-03-24T21:03:05.664ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60992026-03-24T21:03:05.665ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
61002026-03-24T21:03:05.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
61012026-03-24T21:03:05.667ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
61022026-03-24T21:03:05.667ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
61032026-03-24T21:03:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
61042026-03-24T21:03:05.668ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
61052026-03-24T21:03:05.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61062026-03-24T21:03:05.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61072026-03-24T21:03:05.675ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = DeployNexusRecords step id = Ensure
61082026-03-24T21:03:05.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61092026-03-24T21:03:05.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61102026-03-24T21:03:05.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61112026-03-24T21:03:05.681ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
61122026-03-24T21:03:05.681ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
61132026-03-24T21:03:05.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61142026-03-24T21:03:05.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61152026-03-24T21:03:05.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
61162026-03-24T21:03:05.688ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 131.093995ms iteration = 13 status = {"errors":[],"requests_created_ok":[]}
61172026-03-24T21:03:05.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61182026-03-24T21:03:05.689ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
61192026-03-24T21:03:05.689ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
61202026-03-24T21:03:05.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61212026-03-24T21:03:05.689ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 157.25233ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
61222026-03-24T21:03:05.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 136.437349ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
61232026-03-24T21:03:05.693ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 139.210203ms iteration = 13 status = {"errors":[],"finish_invoked_ok":[]}
61242026-03-24T21:03:05.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61252026-03-24T21:03:05.701ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
61262026-03-24T21:03:05.704ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
61272026-03-24T21:03:05.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
61282026-03-24T21:03:05.710ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
61292026-03-24T21:03:05.710ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
61302026-03-24T21:03:05.710ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
61312026-03-24T21:03:05.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
61322026-03-24T21:03:05.711ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = SledAgent step id = Ensure
61332026-03-24T21:03:05.713ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
61342026-03-24T21:03:05.716ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "60314bb9-a4ca-4b8e-bef5-e6738ac0a9d6", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} }) sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
61352026-03-24T21:03:05.717ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = FirewallRules step id = Ensure
61362026-03-24T21:03:05.717ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61372026-03-24T21:03:05.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61382026-03-24T21:03:05.721ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 189.288564ms iteration = 22 status = {"physical_disks_added":0}
61392026-03-24T21:03:05.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61402026-03-24T21:03:05.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61412026-03-24T21:03:05.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61422026-03-24T21:03:05.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61432026-03-24T21:03:05.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61442026-03-24T21:03:05.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61452026-03-24T21:03:05.734ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
61462026-03-24T21:03:05.734ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 202.200662ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
61472026-03-24T21:03:05.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61482026-03-24T21:03:05.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61492026-03-24T21:03:05.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61502026-03-24T21:03:05.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61512026-03-24T21:03:05.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61522026-03-24T21:03:05.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
61532026-03-24T21:03:05.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61542026-03-24T21:03:05.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61552026-03-24T21:03:05.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
61562026-03-24T21:03:05.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61572026-03-24T21:03:05.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
61582026-03-24T21:03:05.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61592026-03-24T21:03:05.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61602026-03-24T21:03:05.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61612026-03-24T21:03:05.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
61622026-03-24T21:03:05.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
61632026-03-24T21:03:05.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61642026-03-24T21:03:05.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61652026-03-24T21:03:05.764ZINFOnexus (BackgroundTasks): Loaded 0 active trust quorum configurations from database background_task = trust_quorum_manager bgtask = TrustQuorumManager file = nexus/src/app/background/tasks/trust_quorum.rs:83
61662026-03-24T21:03:05.764ZDEBGnexus (BackgroundTasks): activation complete background_task = trust_quorum_manager elapsed = 189.022387ms iteration = 7 status = {"PerRackStatus":{"errors":[],"statuses":[]}}
61672026-03-24T21:03:05.767ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
61682026-03-24T21:03:05.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
61692026-03-24T21:03:05.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
61702026-03-24T21:03:05.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
61712026-03-24T21:03:05.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61722026-03-24T21:03:05.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61732026-03-24T21:03:05.769ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 223.205429ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
61742026-03-24T21:03:05.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
61752026-03-24T21:03:05.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
61762026-03-24T21:03:05.778ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2026-03-24T20:56:59.957750Z, time_modified: 2026-03-24T20:56:59.957750Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }) }
61772026-03-24T21:03:05.778ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 257.640398ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]}},"default_endpoint":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
61782026-03-24T21:03:05.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61792026-03-24T21:03:05.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61802026-03-24T21:03:05.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61812026-03-24T21:03:05.781ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
61822026-03-24T21:03:05.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
61832026-03-24T21:03:05.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61842026-03-24T21:03:05.782ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
61852026-03-24T21:03:05.782ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
61862026-03-24T21:03:05.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61872026-03-24T21:03:05.783ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3ed80588-08f8-479e-a320-2bef7761c387", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} })
61882026-03-24T21:03:05.783ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61892026-03-24T21:03:05.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61902026-03-24T21:03:05.784ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
61912026-03-24T21:03:05.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
61922026-03-24T21:03:05.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
61932026-03-24T21:03:05.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61942026-03-24T21:03:05.785ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 231.482331ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
61952026-03-24T21:03:05.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61962026-03-24T21:03:05.786ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
61972026-03-24T21:03:05.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
61982026-03-24T21:03:05.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61992026-03-24T21:03:05.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62002026-03-24T21:03:05.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62012026-03-24T21:03:05.790ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
62022026-03-24T21:03:05.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62032026-03-24T21:03:05.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62042026-03-24T21:03:05.791ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
62052026-03-24T21:03:05.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
62062026-03-24T21:03:05.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62072026-03-24T21:03:05.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62082026-03-24T21:03:05.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62092026-03-24T21:03:05.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62102026-03-24T21:03:05.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62112026-03-24T21:03:05.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62122026-03-24T21:03:05.799ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
62132026-03-24T21:03:05.799ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
62142026-03-24T21:03:05.800ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "19f8ee95-dc7f-4af6-9d16-d9df08c36d49", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} })
62152026-03-24T21:03:05.800ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 245.965002ms iteration = 19 status = {}
62162026-03-24T21:03:05.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62172026-03-24T21:03:05.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62182026-03-24T21:03:05.806ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
62192026-03-24T21:03:05.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62202026-03-24T21:03:05.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62212026-03-24T21:03:05.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62222026-03-24T21:03:05.808ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
62232026-03-24T21:03:05.808ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
62242026-03-24T21:03:05.808ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
62252026-03-24T21:03:05.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62262026-03-24T21:03:05.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
62272026-03-24T21:03:05.810ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 256.187459ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
62282026-03-24T21:03:05.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62292026-03-24T21:03:05.810ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
62302026-03-24T21:03:05.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62312026-03-24T21:03:05.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62322026-03-24T21:03:05.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62332026-03-24T21:03:05.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62342026-03-24T21:03:05.814ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
62352026-03-24T21:03:05.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62362026-03-24T21:03:05.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62372026-03-24T21:03:05.815ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62382026-03-24T21:03:05.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62392026-03-24T21:03:05.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62402026-03-24T21:03:05.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62412026-03-24T21:03:05.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62422026-03-24T21:03:05.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62432026-03-24T21:03:05.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62442026-03-24T21:03:05.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62452026-03-24T21:03:05.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62462026-03-24T21:03:05.822ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
62472026-03-24T21:03:05.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62482026-03-24T21:03:05.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62492026-03-24T21:03:05.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62502026-03-24T21:03:05.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62512026-03-24T21:03:05.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62522026-03-24T21:03:05.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62532026-03-24T21:03:05.829ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
62542026-03-24T21:03:05.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
62552026-03-24T21:03:05.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62562026-03-24T21:03:05.833ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
62572026-03-24T21:03:05.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
62582026-03-24T21:03:05.836ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62592026-03-24T21:03:05.838ZDEBGnexus (BackgroundTasks): listed attached subnets background_task = attached_subnet_manager n_subnets = 0
62602026-03-24T21:03:05.838ZDEBGnexus (BackgroundTasks): activation complete background_task = attached_subnet_manager elapsed = 263.535136ms iteration = 7 status = {"db_error":null,"dendrite":{"switch0":{"errors":[],"n_subnets_added":0,"n_subnets_removed":0,"n_total_subnets":0}},"sled":{}}
62612026-03-24T21:03:05.839ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
62622026-03-24T21:03:05.839ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
62632026-03-24T21:03:05.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
62642026-03-24T21:03:05.840ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
62652026-03-24T21:03:05.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62662026-03-24T21:03:05.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62672026-03-24T21:03:05.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62682026-03-24T21:03:05.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62692026-03-24T21:03:05.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62702026-03-24T21:03:05.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62712026-03-24T21:03:05.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62722026-03-24T21:03:05.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62732026-03-24T21:03:05.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62742026-03-24T21:03:05.851ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
62752026-03-24T21:03:05.852ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "7a702af3-3782-499e-96c9-c26a64e8faa4", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} })
62762026-03-24T21:03:05.852ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
62772026-03-24T21:03:05.852ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
62782026-03-24T21:03:05.852ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 277.562885ms iteration = 7 status = {"errors":[],"probes_by_sled":{"b4293d4f-3308-4dfa-b261-a103215913e4":0}}
62792026-03-24T21:03:05.852ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Dependency
62802026-03-24T21:03:05.852ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
62812026-03-24T21:03:05.852ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
62822026-03-24T21:03:05.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
62832026-03-24T21:03:05.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62842026-03-24T21:03:05.854ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
62852026-03-24T21:03:05.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62862026-03-24T21:03:05.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62872026-03-24T21:03:05.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62882026-03-24T21:03:05.855ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62892026-03-24T21:03:05.859ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
62902026-03-24T21:03:05.859ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
62912026-03-24T21:03:05.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
62922026-03-24T21:03:05.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62932026-03-24T21:03:05.862ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62942026-03-24T21:03:05.863ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
62952026-03-24T21:03:05.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
62962026-03-24T21:03:05.864ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
62972026-03-24T21:03:05.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62982026-03-24T21:03:05.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62992026-03-24T21:03:05.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63002026-03-24T21:03:05.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63012026-03-24T21:03:05.866ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
63022026-03-24T21:03:05.866ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63032026-03-24T21:03:05.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63042026-03-24T21:03:05.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63052026-03-24T21:03:05.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63062026-03-24T21:03:05.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63072026-03-24T21:03:05.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63082026-03-24T21:03:05.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63092026-03-24T21:03:05.873ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
63102026-03-24T21:03:05.873ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
63112026-03-24T21:03:05.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63122026-03-24T21:03:05.876ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63132026-03-24T21:03:05.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63142026-03-24T21:03:05.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63152026-03-24T21:03:05.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63162026-03-24T21:03:05.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63172026-03-24T21:03:05.879ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fbc6ad7-7b2c-4b35-8337-72b3da234703", "content-length": "372", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} })
63182026-03-24T21:03:05.879ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63192026-03-24T21:03:05.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63202026-03-24T21:03:05.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63212026-03-24T21:03:05.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63222026-03-24T21:03:05.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63232026-03-24T21:03:05.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63242026-03-24T21:03:05.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63252026-03-24T21:03:05.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63262026-03-24T21:03:05.884ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
63272026-03-24T21:03:05.884ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
63282026-03-24T21:03:05.884ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 330.403139ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
63292026-03-24T21:03:05.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63302026-03-24T21:03:05.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63312026-03-24T21:03:05.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63322026-03-24T21:03:05.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63332026-03-24T21:03:05.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63342026-03-24T21:03:05.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63352026-03-24T21:03:05.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63362026-03-24T21:03:05.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63372026-03-24T21:03:05.894ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
63382026-03-24T21:03:05.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63392026-03-24T21:03:05.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63402026-03-24T21:03:05.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63412026-03-24T21:03:05.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63422026-03-24T21:03:05.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63432026-03-24T21:03:05.899ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
63442026-03-24T21:03:05.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63452026-03-24T21:03:05.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63462026-03-24T21:03:05.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63472026-03-24T21:03:05.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63482026-03-24T21:03:05.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63492026-03-24T21:03:05.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63502026-03-24T21:03:05.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63512026-03-24T21:03:05.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63522026-03-24T21:03:05.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63532026-03-24T21:03:05.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63542026-03-24T21:03:05.907ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
63552026-03-24T21:03:05.907ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
63562026-03-24T21:03:05.907ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a3a4ce9a-2cd8-4b19-80ca-787021816b5f", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} })
63572026-03-24T21:03:05.907ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63582026-03-24T21:03:05.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63592026-03-24T21:03:05.911ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63602026-03-24T21:03:05.911ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
63612026-03-24T21:03:05.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63622026-03-24T21:03:05.912ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
63632026-03-24T21:03:05.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63642026-03-24T21:03:05.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63652026-03-24T21:03:05.914ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63662026-03-24T21:03:05.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63672026-03-24T21:03:05.915ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63682026-03-24T21:03:05.916ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0f32c3d5-0b77-4ead-b4f8-856453ed27f6", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} })
63692026-03-24T21:03:05.916ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
63702026-03-24T21:03:05.916ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
63712026-03-24T21:03:05.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
63722026-03-24T21:03:05.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63732026-03-24T21:03:05.918ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
63742026-03-24T21:03:05.918ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
63752026-03-24T21:03:05.919ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63762026-03-24T21:03:05.923ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
63772026-03-24T21:03:05.923ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63782026-03-24T21:03:05.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
63792026-03-24T21:03:05.924ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
63802026-03-24T21:03:05.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63812026-03-24T21:03:05.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63822026-03-24T21:03:05.925ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63832026-03-24T21:03:05.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63842026-03-24T21:03:05.926ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c292cc37-1542-4c76-88ba-98148f977428", "date": "Tue, 24 Mar 2026 21:03:05 GMT"} })
63852026-03-24T21:03:05.926ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 73.280836ms iteration = 20 status = {}
63862026-03-24T21:03:05.928ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
63872026-03-24T21:03:05.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63882026-03-24T21:03:05.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63892026-03-24T21:03:05.928ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
63902026-03-24T21:03:05.928ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63912026-03-24T21:03:05.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
63922026-03-24T21:03:05.930ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
63932026-03-24T21:03:05.930ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
63942026-03-24T21:03:05.930ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
63952026-03-24T21:03:05.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63962026-03-24T21:03:05.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63972026-03-24T21:03:05.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
63982026-03-24T21:03:05.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
63992026-03-24T21:03:05.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64002026-03-24T21:03:05.982ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
64012026-03-24T21:03:05.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64022026-03-24T21:03:05.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64032026-03-24T21:03:05.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64042026-03-24T21:03:05.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64052026-03-24T21:03:05.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64062026-03-24T21:03:05.992ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
64072026-03-24T21:03:05.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64082026-03-24T21:03:05.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64092026-03-24T21:03:05.994ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64102026-03-24T21:03:05.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64112026-03-24T21:03:05.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64122026-03-24T21:03:05.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64132026-03-24T21:03:05.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64142026-03-24T21:03:05.997ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64152026-03-24T21:03:05.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64162026-03-24T21:03:06.000ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64172026-03-24T21:03:06.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64182026-03-24T21:03:06.001ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
64192026-03-24T21:03:06.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64202026-03-24T21:03:06.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64212026-03-24T21:03:06.002ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 470.282961ms iteration = 13 status = {}
64222026-03-24T21:03:06.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64232026-03-24T21:03:06.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64242026-03-24T21:03:06.011ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64252026-03-24T21:03:06.014ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
64262026-03-24T21:03:06.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64272026-03-24T21:03:06.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64282026-03-24T21:03:06.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64292026-03-24T21:03:06.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64302026-03-24T21:03:06.018ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64312026-03-24T21:03:06.020ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
64322026-03-24T21:03:06.020ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64332026-03-24T21:03:06.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64342026-03-24T21:03:06.027ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
64352026-03-24T21:03:06.027ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
64362026-03-24T21:03:06.027ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = ffc8d9e0-4ca4-49ef-a3e9-72ffbac44064 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
64372026-03-24T21:03:06.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64382026-03-24T21:03:06.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64392026-03-24T21:03:06.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64402026-03-24T21:03:06.033ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
64412026-03-24T21:03:06.033ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64422026-03-24T21:03:06.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64432026-03-24T21:03:06.037ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
64442026-03-24T21:03:06.037ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
64452026-03-24T21:03:06.037ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 505.546372ms iteration = 18 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
64462026-03-24T21:03:06.042ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
64472026-03-24T21:03:06.042ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
64482026-03-24T21:03:06.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64492026-03-24T21:03:06.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64502026-03-24T21:03:06.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64512026-03-24T21:03:06.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64522026-03-24T21:03:06.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64532026-03-24T21:03:06.053ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
64542026-03-24T21:03:06.053ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
64552026-03-24T21:03:06.054ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "ab4e9d40-94f6-43ee-83c6-819b4c4b44a9", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
64562026-03-24T21:03:06.054ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
64572026-03-24T21:03:06.054ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = Dns step id = Ensure
64582026-03-24T21:03:06.054ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
64592026-03-24T21:03:06.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64602026-03-24T21:03:06.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64612026-03-24T21:03:06.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64622026-03-24T21:03:06.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64632026-03-24T21:03:06.057ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:118 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
64642026-03-24T21:03:06.057ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 502.86735ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
64652026-03-24T21:03:06.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64662026-03-24T21:03:06.061ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
64672026-03-24T21:03:06.061ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
64682026-03-24T21:03:06.062ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f29391e8-213a-4522-a196-d90f0e3ebbe7", "content-length": "691", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
64692026-03-24T21:03:06.062ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
64702026-03-24T21:03:06.062ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
64712026-03-24T21:03:06.062ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
64722026-03-24T21:03:06.062ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 530.399672ms iteration = 13 status = {}
64732026-03-24T21:03:06.062ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
64742026-03-24T21:03:06.062ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
64752026-03-24T21:03:06.062ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
64762026-03-24T21:03:06.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64772026-03-24T21:03:06.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64782026-03-24T21:03:06.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64792026-03-24T21:03:06.068ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
64802026-03-24T21:03:06.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64812026-03-24T21:03:06.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64822026-03-24T21:03:06.070ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
64832026-03-24T21:03:06.070ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
64842026-03-24T21:03:06.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64852026-03-24T21:03:06.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64862026-03-24T21:03:06.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64872026-03-24T21:03:06.075ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
64882026-03-24T21:03:06.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64892026-03-24T21:03:06.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64902026-03-24T21:03:06.078ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
64912026-03-24T21:03:06.078ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
64922026-03-24T21:03:06.078ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
64932026-03-24T21:03:06.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64942026-03-24T21:03:06.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64952026-03-24T21:03:06.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64962026-03-24T21:03:06.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64972026-03-24T21:03:06.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64982026-03-24T21:03:06.084ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
64992026-03-24T21:03:06.085ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
65002026-03-24T21:03:06.085ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
65012026-03-24T21:03:06.085ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
65022026-03-24T21:03:06.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65032026-03-24T21:03:06.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65042026-03-24T21:03:06.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65052026-03-24T21:03:06.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
65062026-03-24T21:03:06.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65072026-03-24T21:03:06.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65082026-03-24T21:03:06.094ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
65092026-03-24T21:03:06.094ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
65102026-03-24T21:03:06.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65112026-03-24T21:03:06.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65122026-03-24T21:03:06.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65132026-03-24T21:03:06.100ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
65142026-03-24T21:03:06.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65152026-03-24T21:03:06.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65162026-03-24T21:03:06.104ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
65172026-03-24T21:03:06.104ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
65182026-03-24T21:03:06.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65192026-03-24T21:03:06.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65202026-03-24T21:03:06.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65212026-03-24T21:03:06.108ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
65222026-03-24T21:03:06.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65232026-03-24T21:03:06.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65242026-03-24T21:03:06.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65252026-03-24T21:03:06.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65262026-03-24T21:03:06.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65272026-03-24T21:03:06.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65282026-03-24T21:03:06.113ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65292026-03-24T21:03:06.116ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
65302026-03-24T21:03:06.117ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
65312026-03-24T21:03:06.117ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
65322026-03-24T21:03:06.117ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = OmicronZones step id = Cleanup
65332026-03-24T21:03:06.117ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = OmicronZones step id = Cleanup
65342026-03-24T21:03:06.117ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = PhysicalDisks step id = Cleanup
65352026-03-24T21:03:06.117ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = Clickhouse step id = Ensure
65362026-03-24T21:03:06.117ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = Clickhouse step id = Ensure
65372026-03-24T21:03:06.118ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
65382026-03-24T21:03:06.123ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "dad84d9b-8f35-411a-93ca-a347661d9744", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
65392026-03-24T21:03:06.123ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = SupportBundles step id = Cleanup
65402026-03-24T21:03:06.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65412026-03-24T21:03:06.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65422026-03-24T21:03:06.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65432026-03-24T21:03:06.127ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
65442026-03-24T21:03:06.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65452026-03-24T21:03:06.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65462026-03-24T21:03:06.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65472026-03-24T21:03:06.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65482026-03-24T21:03:06.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65492026-03-24T21:03:06.133ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
65502026-03-24T21:03:06.136ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = OmicronZones step id = Cleanup
65512026-03-24T21:03:06.136ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
65522026-03-24T21:03:06.136ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
65532026-03-24T21:03:06.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65542026-03-24T21:03:06.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65552026-03-24T21:03:06.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65562026-03-24T21:03:06.140ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
65572026-03-24T21:03:06.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65582026-03-24T21:03:06.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65592026-03-24T21:03:06.141ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
65602026-03-24T21:03:06.141ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9e1c09c1-3960-4ecf-a904-19e137699ab3 (blueprint), false)
65612026-03-24T21:03:06.141ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = Cockroach step id = Ensure
65622026-03-24T21:03:06.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65632026-03-24T21:03:06.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65642026-03-24T21:03:06.164ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
65652026-03-24T21:03:06.164ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ffbbd8f2-b401-46a2-9529-ccb81eb05853 step component = MgsUpdates step id = Ensure
65662026-03-24T21:03:06.164ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 9 reason = Dependency
65672026-03-24T21:03:06.164ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Dependency
65682026-03-24T21:03:06.164ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65692026-03-24T21:03:06.164ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
65702026-03-24T21:03:06.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
65712026-03-24T21:03:06.164ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 643.523302ms iteration = 8 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3140,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":93229014,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":93228964,"secs":0}},"event_index":1,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":93242394,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":60862061,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":60862021,"secs":0}},"event_index":2,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":154124076,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":35956509,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":35956459,"secs":0}},"event_index":3,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":190102226,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5904279,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5904229,"secs":0}},"event_index":4,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":196030056,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":337660105,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":337660065,"secs":0}},"event_index":5,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":533702781,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":63248055,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":63248015,"secs":0}},"event_index":6,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":596961177,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6941,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":6910,"secs":0}},"event_index":7,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":596979657,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2291,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2261,"secs":0}},"event_index":8,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":596983758,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4990,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4960,"secs":0}},"event_index":9,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":596990658,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1870,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1830,"secs":0}},"event_index":10,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":596993888,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5924750,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":5924720,"secs":0}},"event_index":11,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":602919758,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":12814316,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":12814265,"secs":0}},"event_index":12,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":615745783,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4904592,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4904562,"secs":0}},"event_index":13,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":620659916,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":22429095,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":22429055,"secs":0}},"event_index":14,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":643101821,"secs":0}},{"data":{"attempt_elapsed":{"nanos":5560,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5531,"secs":0}},"event_index":15,"execution_id":"05b0ce33-349b-4b68-a516-a09bea8f9a69","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":643115992,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3"}
65722026-03-24T21:03:06.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
65732026-03-24T21:03:06.165ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65742026-03-24T21:03:06.166ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65752026-03-24T21:03:06.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
65762026-03-24T21:03:06.168ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65772026-03-24T21:03:06.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
65782026-03-24T21:03:06.169ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
65792026-03-24T21:03:06.169ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
65802026-03-24T21:03:06.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
65812026-03-24T21:03:06.170ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65822026-03-24T21:03:06.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
65832026-03-24T21:03:06.171ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(8e786f99-51a8-4eb4-98a7-4302327a1608, 0), (1b4f62ac-a6d9-4eb1-9a06-e4cde54a1439, 0), (29a1b98a-4509-4d5f-a750-afad7bceeaaa, 0), (3d19abfd-8175-4fec-8eaa-cfb3a4c435c7, 0)] collection_id = 8e786f99-51a8-4eb4-98a7-4302327a1608
65842026-03-24T21:03:06.171ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
65852026-03-24T21:03:06.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
65862026-03-24T21:03:06.172ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65872026-03-24T21:03:06.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
65882026-03-24T21:03:06.174ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65892026-03-24T21:03:06.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
65902026-03-24T21:03:06.176ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
65912026-03-24T21:03:06.176ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.092666ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
65922026-03-24T21:03:06.217ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 2
65932026-03-24T21:03:06.217ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221
65942026-03-24T21:03:06.218ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
65952026-03-24T21:03:06.218ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:40264
65962026-03-24T21:03:06.218ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:40264 req_id = 0fe0f368-f093-455c-9b0d-ed8273973921 uri = /metrics/producers version = 1.0.0
65972026-03-24T21:03:06.218ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:40264 req_id = 0fe0f368-f093-455c-9b0d-ed8273973921 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
65982026-03-24T21:03:06.219ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:40264 req_id = 0fe0f368-f093-455c-9b0d-ed8273973921 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
65992026-03-24T21:03:06.278ZINFOnexus: assigned collector to new producer collector_id = a309df7c-2a20-4201-8e80-eb9f7e355177 file = nexus/src/app/oximeter.rs:76 producer_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb
66002026-03-24T21:03:06.278ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 60189 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:40264 req_id = 0fe0f368-f093-455c-9b0d-ed8273973921 response_code = 201 uri = /metrics/producers
66012026-03-24T21:03:06.279ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "0fe0f368-f093-455c-9b0d-ed8273973921", "content-length": "41", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66022026-03-24T21:03:06.279ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
66032026-03-24T21:03:06.316ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 8e786f99-51a8-4eb4-98a7-4302327a1608 file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
66042026-03-24T21:03:06.316ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66052026-03-24T21:03:06.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
66062026-03-24T21:03:06.322ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(1b4f62ac-a6d9-4eb1-9a06-e4cde54a1439, 0), (29a1b98a-4509-4d5f-a750-afad7bceeaaa, 0), (3d19abfd-8175-4fec-8eaa-cfb3a4c435c7, 0), (6d53e1e9-ea3f-4511-bee9-394108b97e31, 0)] collection_id = 1b4f62ac-a6d9-4eb1-9a06-e4cde54a1439
66072026-03-24T21:03:06.322ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66082026-03-24T21:03:06.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
66092026-03-24T21:03:06.479ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 1b4f62ac-a6d9-4eb1-9a06-e4cde54a1439 file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
66102026-03-24T21:03:06.479ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66112026-03-24T21:03:06.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
66122026-03-24T21:03:06.484ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(29a1b98a-4509-4d5f-a750-afad7bceeaaa, 0), (3d19abfd-8175-4fec-8eaa-cfb3a4c435c7, 0), (6d53e1e9-ea3f-4511-bee9-394108b97e31, 0), (9789333b-5bf2-4c58-8e77-63169dbd8708, 0)] collection_id = 29a1b98a-4509-4d5f-a750-afad7bceeaaa
66132026-03-24T21:03:06.484ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66142026-03-24T21:03:06.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
66152026-03-24T21:03:06.627ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 29a1b98a-4509-4d5f-a750-afad7bceeaaa file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
66162026-03-24T21:03:06.627ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66172026-03-24T21:03:06.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
66182026-03-24T21:03:06.633ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(3d19abfd-8175-4fec-8eaa-cfb3a4c435c7, 0), (6d53e1e9-ea3f-4511-bee9-394108b97e31, 0), (9789333b-5bf2-4c58-8e77-63169dbd8708, 0)]
66192026-03-24T21:03:06.634ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1172, tv_nsec: 849011483 } })
66202026-03-24T21:03:06.636ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1345db7a-abc1-4a4c-a49c-e6c56102e521.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1920db37-87f5-4a96-b3ef-1db2eb59e9a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6b02d2eb-4373-4f12-8efc-4c0e3f4fa77b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("709c4704-f768-4640-a198-a23621ed6e11.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1172, tv_nsec: 850826032 } })
66212026-03-24T21:03:06.637ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
66222026-03-24T21:03:06.637ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
66232026-03-24T21:03:06.637ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
66242026-03-24T21:03:06.637ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf7fa90c-71c1-45a9-909e-d776b6b8803c", "content-length": "476", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66252026-03-24T21:03:06.637ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
66262026-03-24T21:03:06.638ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0e8220a-eabe-400d-af12-684ff20ba75b", "content-length": "734", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66272026-03-24T21:03:06.638ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
66282026-03-24T21:03:06.638ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a3ae05f-3d0f-40b8-8233-af9b5a126999", "content-length": "111", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66292026-03-24T21:03:06.638ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
66302026-03-24T21:03:06.639ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c56ee54-2e24-4743-bea8-07ec4e2f755a", "content-length": "111", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66312026-03-24T21:03:06.639ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
66322026-03-24T21:03:06.639ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "89eea8d3-f43b-40ad-b5a1-b7ef3157e087", "content-length": "163", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66332026-03-24T21:03:06.639ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
66342026-03-24T21:03:06.640ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "62e077a2-e88f-4a7b-a744-21b495689a2d", "content-length": "163", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66352026-03-24T21:03:06.640ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
66362026-03-24T21:03:06.640ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "edc7696e-313d-4701-a1da-0987d5ca1c63", "content-length": "166", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66372026-03-24T21:03:06.640ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
66382026-03-24T21:03:06.641ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4297acbb-acb3-43dc-a24a-ad62867ec595", "content-length": "166", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66392026-03-24T21:03:06.641ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
66402026-03-24T21:03:06.641ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "59580b48-85fc-4caa-8714-bf517dd817a5", "content-length": "702", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66412026-03-24T21:03:06.641ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
66422026-03-24T21:03:06.641ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "41df9faf-ddf0-4b1d-8661-9ca2d0213ced", "content-length": "727", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66432026-03-24T21:03:06.641ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
66442026-03-24T21:03:06.642ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a04d91b6-ea97-4092-aaef-9d6adafca4df", "content-length": "729", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66452026-03-24T21:03:06.642ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
66462026-03-24T21:03:06.642ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "93bda049-e003-4c83-badd-e1c6f3fe7368", "content-length": "728", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66472026-03-24T21:03:06.642ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
66482026-03-24T21:03:06.643ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d3e500c-97ad-4eb5-a074-4f9430942ff8", "content-length": "722", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66492026-03-24T21:03:06.643ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
66502026-03-24T21:03:06.643ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "160f8349-90f2-4393-8d18-23e4fcf7a842", "content-length": "10", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66512026-03-24T21:03:06.644ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
66522026-03-24T21:03:06.647ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c4d5550-05ff-4b7e-be39-4c6e753ce53c", "content-length": "144", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66532026-03-24T21:03:06.647ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
66542026-03-24T21:03:06.647ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "972bfaf6-2160-46ea-882a-e3fb32b7b0d1", "content-length": "147", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66552026-03-24T21:03:06.647ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
66562026-03-24T21:03:06.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4c08739-7003-4644-bc9c-50df515f065b", "content-length": "109", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66572026-03-24T21:03:06.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
66582026-03-24T21:03:06.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "24d1b89b-0d95-4638-a436-3266334ca182", "content-length": "109", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66592026-03-24T21:03:06.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
66602026-03-24T21:03:06.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "544a2617-ad44-42b1-98d2-3596b0fc28c3", "content-length": "163", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66612026-03-24T21:03:06.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
66622026-03-24T21:03:06.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b886d92-8cde-4083-a80e-a0d0720826fb", "content-length": "163", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66632026-03-24T21:03:06.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
66642026-03-24T21:03:06.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a0f74c0-b096-4d5e-8a8a-3cd0ac8b7a9c", "content-length": "166", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66652026-03-24T21:03:06.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
66662026-03-24T21:03:06.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "53a59ea9-77fc-4f6e-9da7-ef729b98e71f", "content-length": "166", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66672026-03-24T21:03:06.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
66682026-03-24T21:03:06.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9040755-1d6c-474a-b6fd-e8db933103d0", "content-length": "702", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66692026-03-24T21:03:06.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
66702026-03-24T21:03:06.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b320b81-0dd5-4b40-ad31-ca8743e80981", "content-length": "727", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66712026-03-24T21:03:06.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
66722026-03-24T21:03:06.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e85842b7-b434-4b28-a86c-22d4995f0db7", "content-length": "729", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66732026-03-24T21:03:06.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
66742026-03-24T21:03:06.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b4d43ef-5cb6-4340-8ca7-9591953b92bc", "content-length": "728", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66752026-03-24T21:03:06.651ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66762026-03-24T21:03:06.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
66772026-03-24T21:03:06.653ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66782026-03-24T21:03:06.655ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
66792026-03-24T21:03:06.655ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66802026-03-24T21:03:06.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
66812026-03-24T21:03:06.657ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66822026-03-24T21:03:06.659ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
66832026-03-24T21:03:06.659ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66842026-03-24T21:03:06.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
66852026-03-24T21:03:06.663ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
66862026-03-24T21:03:06.663ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
66872026-03-24T21:03:06.721ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "87068698-df01-45bd-8c2d-b814ecf79c64", "content-length": "110670", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66882026-03-24T21:03:06.722ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
66892026-03-24T21:03:06.722ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
66902026-03-24T21:03:06.722ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
66912026-03-24T21:03:06.722ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
66922026-03-24T21:03:06.722ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
66932026-03-24T21:03:06.722ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
66942026-03-24T21:03:06.722ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
66952026-03-24T21:03:06.722ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
66962026-03-24T21:03:06.722ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "295b999f-316d-4072-9918-a6e113305588", "content-length": "64", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66972026-03-24T21:03:06.722ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
66982026-03-24T21:03:06.722ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e401a1f9-4b7d-436f-b939-93870386f060", "content-length": "64", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
66992026-03-24T21:03:06.723ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a82a195-014f-45d1-b1a6-917c940f95fa", "content-length": "64", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67002026-03-24T21:03:06.723ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "47b2700e-e81a-4cad-a61e-079493b41682", "content-length": "64", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67012026-03-24T21:03:06.723ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
67022026-03-24T21:03:06.723ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
67032026-03-24T21:03:06.723ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
67042026-03-24T21:03:06.723ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "30a4830f-1718-46df-a332-cc30bf5c923f", "content-length": "64", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67052026-03-24T21:03:06.723ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
67062026-03-24T21:03:06.730ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f33bd503-3922-473c-af95-f5c9fa2de510", "content-length": "389917", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67072026-03-24T21:03:06.730ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "725b4396-081b-49af-bc6f-68592ce0def0", "content-length": "400663", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67082026-03-24T21:03:06.730ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "17831d49-bceb-4558-8f01-ef02896b9504", "content-length": "394094", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67092026-03-24T21:03:06.730ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb3af866-3712-4963-b3c1-bf721887dd26", "content-length": "395027", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67102026-03-24T21:03:06.730ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c02f007-8153-4773-be70-0fb322509a5d", "content-length": "425901", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67112026-03-24T21:03:06.734ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
67122026-03-24T21:03:06.734ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
67132026-03-24T21:03:06.734ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
67142026-03-24T21:03:06.734ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
67152026-03-24T21:03:06.734ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
67162026-03-24T21:03:06.735ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 (omicron_zone)
67172026-03-24T21:03:06.735ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
67182026-03-24T21:03:06.739ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "f74de04a-5c4c-472d-b0bb-627bb0ca189b", "content-length": "128", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67192026-03-24T21:03:06.739ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
67202026-03-24T21:03:06.741ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
67212026-03-24T21:03:06.741ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
67222026-03-24T21:03:06.742ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "983fb8a6-c1f8-404e-b707-5be3e7fc0c67", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67232026-03-24T21:03:06.742ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
67242026-03-24T21:03:06.742ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
67252026-03-24T21:03:06.742ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
67262026-03-24T21:03:06.743ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7ffa4c2-5a42-41d5-88b5-43f3e70e8b4a", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67272026-03-24T21:03:06.743ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
67282026-03-24T21:03:06.743ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
67292026-03-24T21:03:06.743ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
67302026-03-24T21:03:06.743ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "09927a46-4520-4882-a7e1-db53fe8c8bb6", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:03:06 GMT"} })
67312026-03-24T21:03:06.743ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
67322026-03-24T21:03:06.743ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
67332026-03-24T21:03:06.743ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
67342026-03-24T21:03:06.743ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
67352026-03-24T21:03:06.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
67362026-03-24T21:03:06.746ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67372026-03-24T21:03:06.748ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
67382026-03-24T21:03:06.748ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
67392026-03-24T21:03:06.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
67402026-03-24T21:03:06.828ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 32f38600-4055-4a7f-bcbc-7299e531f9f7 file = nexus/db-queries/src/db/datastore/inventory.rs:1757
67412026-03-24T21:03:06.828ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 32f38600-4055-4a7f-bcbc-7299e531f9f7 time_started = 2026-03-24 21:03:06.637024 UTC
67422026-03-24T21:03:06.828ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 34 reason = Dependency
67432026-03-24T21:03:06.828ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
67442026-03-24T21:03:06.828ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 664.022385ms iteration = 9 status = {"collection_id":"32f38600-4055-4a7f-bcbc-7299e531f9f7","time_done":"2026-03-24 21:03:06.743912 UTC","time_started":"2026-03-24 21:03:06.637024 UTC"}
67452026-03-24T21:03:06.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
67462026-03-24T21:03:06.830ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67472026-03-24T21:03:06.833ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
67482026-03-24T21:03:06.833ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
67492026-03-24T21:03:06.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
67502026-03-24T21:03:07.108ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
67512026-03-24T21:03:07.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
67522026-03-24T21:03:07.165ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 337.775868ms iteration = 34 status = {"Loaded":{"collection_id":"bfcfe115-13a5-4646-a408-f5f6ef741c06","time_loaded":"2026-03-24T21:03:06.828163293Z","time_started":"2026-03-24T21:03:06.651617Z"}}
67532026-03-24T21:03:07.166ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Dependency
67542026-03-24T21:03:07.166ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 12 reason = Dependency
67552026-03-24T21:03:07.166ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 19 reason = Dependency
67562026-03-24T21:03:07.166ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 17 reason = Dependency
67572026-03-24T21:03:07.166ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
67582026-03-24T21:03:07.166ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 9.07\u{b5}s iteration = 17 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
67592026-03-24T21:03:07.166ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67602026-03-24T21:03:07.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67612026-03-24T21:03:07.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
67622026-03-24T21:03:07.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
67632026-03-24T21:03:07.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67642026-03-24T21:03:07.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67652026-03-24T21:03:07.168ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67662026-03-24T21:03:07.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67672026-03-24T21:03:07.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67682026-03-24T21:03:07.172ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
67692026-03-24T21:03:07.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
67702026-03-24T21:03:07.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
67712026-03-24T21:03:07.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
67722026-03-24T21:03:07.172ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67732026-03-24T21:03:07.173ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
67742026-03-24T21:03:07.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67752026-03-24T21:03:07.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67762026-03-24T21:03:07.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67772026-03-24T21:03:07.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67782026-03-24T21:03:07.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67792026-03-24T21:03:07.175ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67802026-03-24T21:03:07.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67812026-03-24T21:03:07.177ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
67822026-03-24T21:03:07.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
67832026-03-24T21:03:07.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
67842026-03-24T21:03:07.177ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67852026-03-24T21:03:07.177ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
67862026-03-24T21:03:07.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67872026-03-24T21:03:07.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67882026-03-24T21:03:07.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
67892026-03-24T21:03:07.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
67902026-03-24T21:03:07.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
67912026-03-24T21:03:07.179ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
67922026-03-24T21:03:07.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
67932026-03-24T21:03:07.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67942026-03-24T21:03:07.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67952026-03-24T21:03:07.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67962026-03-24T21:03:07.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67972026-03-24T21:03:07.183ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
67982026-03-24T21:03:07.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
67992026-03-24T21:03:07.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
68002026-03-24T21:03:07.184ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
68012026-03-24T21:03:07.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68022026-03-24T21:03:07.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68032026-03-24T21:03:07.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68042026-03-24T21:03:07.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68052026-03-24T21:03:07.188ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 22.340933ms iteration = 23 status = {"physical_disks_added":0}
68062026-03-24T21:03:07.188ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
68072026-03-24T21:03:07.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
68082026-03-24T21:03:07.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
68092026-03-24T21:03:07.189ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
68102026-03-24T21:03:07.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68112026-03-24T21:03:07.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68122026-03-24T21:03:07.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68132026-03-24T21:03:07.192ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
68142026-03-24T21:03:07.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
68152026-03-24T21:03:07.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68162026-03-24T21:03:07.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
68172026-03-24T21:03:07.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68182026-03-24T21:03:07.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68192026-03-24T21:03:07.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68202026-03-24T21:03:07.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68212026-03-24T21:03:07.196ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
68222026-03-24T21:03:07.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
68232026-03-24T21:03:07.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
68242026-03-24T21:03:07.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68252026-03-24T21:03:07.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
68262026-03-24T21:03:07.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68272026-03-24T21:03:07.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68282026-03-24T21:03:07.200ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
68292026-03-24T21:03:07.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
68302026-03-24T21:03:07.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
68312026-03-24T21:03:07.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68322026-03-24T21:03:07.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68332026-03-24T21:03:07.202ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
68342026-03-24T21:03:07.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
68352026-03-24T21:03:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
68362026-03-24T21:03:07.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68372026-03-24T21:03:07.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68382026-03-24T21:03:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
68392026-03-24T21:03:07.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68402026-03-24T21:03:07.206ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
68412026-03-24T21:03:07.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
68422026-03-24T21:03:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68432026-03-24T21:03:07.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
68442026-03-24T21:03:07.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68452026-03-24T21:03:07.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68462026-03-24T21:03:07.210ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
68472026-03-24T21:03:07.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
68482026-03-24T21:03:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
68492026-03-24T21:03:07.211ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
68502026-03-24T21:03:07.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68512026-03-24T21:03:07.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68522026-03-24T21:03:07.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68532026-03-24T21:03:07.212ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
68542026-03-24T21:03:07.212ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 46.833203ms iteration = 12 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"bfcfe115-13a5-4646-a408-f5f6ef741c06","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
68552026-03-24T21:03:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68562026-03-24T21:03:07.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68572026-03-24T21:03:07.216ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
68582026-03-24T21:03:07.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68592026-03-24T21:03:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68602026-03-24T21:03:07.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68612026-03-24T21:03:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68622026-03-24T21:03:07.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68632026-03-24T21:03:07.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
68642026-03-24T21:03:07.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68652026-03-24T21:03:07.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68662026-03-24T21:03:07.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68672026-03-24T21:03:07.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68682026-03-24T21:03:07.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68692026-03-24T21:03:07.231ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
68702026-03-24T21:03:07.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68712026-03-24T21:03:07.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68722026-03-24T21:03:07.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68732026-03-24T21:03:07.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68742026-03-24T21:03:07.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68752026-03-24T21:03:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68762026-03-24T21:03:07.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68772026-03-24T21:03:07.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
68782026-03-24T21:03:07.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68792026-03-24T21:03:07.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68802026-03-24T21:03:07.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68812026-03-24T21:03:07.261ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
68822026-03-24T21:03:07.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68832026-03-24T21:03:07.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68842026-03-24T21:03:07.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68852026-03-24T21:03:07.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68862026-03-24T21:03:07.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68872026-03-24T21:03:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68882026-03-24T21:03:07.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68892026-03-24T21:03:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
68902026-03-24T21:03:07.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68912026-03-24T21:03:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68922026-03-24T21:03:07.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68932026-03-24T21:03:07.273ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
68942026-03-24T21:03:07.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68952026-03-24T21:03:07.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68962026-03-24T21:03:07.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68972026-03-24T21:03:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68982026-03-24T21:03:07.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68992026-03-24T21:03:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69002026-03-24T21:03:07.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69012026-03-24T21:03:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
69022026-03-24T21:03:07.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69032026-03-24T21:03:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69042026-03-24T21:03:07.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69052026-03-24T21:03:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69062026-03-24T21:03:07.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69072026-03-24T21:03:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
69082026-03-24T21:03:07.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69092026-03-24T21:03:07.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69102026-03-24T21:03:07.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69112026-03-24T21:03:07.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69122026-03-24T21:03:07.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69132026-03-24T21:03:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
69142026-03-24T21:03:07.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69152026-03-24T21:03:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69162026-03-24T21:03:07.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69172026-03-24T21:03:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69182026-03-24T21:03:07.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69192026-03-24T21:03:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
69202026-03-24T21:03:07.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69212026-03-24T21:03:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69222026-03-24T21:03:07.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69232026-03-24T21:03:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69242026-03-24T21:03:07.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69252026-03-24T21:03:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
69262026-03-24T21:03:07.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69272026-03-24T21:03:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69282026-03-24T21:03:07.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69292026-03-24T21:03:07.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69302026-03-24T21:03:07.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69312026-03-24T21:03:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
69322026-03-24T21:03:07.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69332026-03-24T21:03:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69342026-03-24T21:03:07.312ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
69352026-03-24T21:03:07.312ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
69362026-03-24T21:03:07.312ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = e21eb688-c337-40c4-bc82-9986003a0b28 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
69372026-03-24T21:03:07.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69382026-03-24T21:03:07.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69392026-03-24T21:03:07.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69402026-03-24T21:03:07.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
69412026-03-24T21:03:07.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69422026-03-24T21:03:07.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69432026-03-24T21:03:07.319ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
69442026-03-24T21:03:07.319ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
69452026-03-24T21:03:07.319ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 153.674323ms iteration = 19 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
69462026-03-24T21:03:08.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 382 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 5142ef28-6355-4cc1-8c2f-7abdb426ac67 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
69472026-03-24T21:03:10.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 74 reason = Timeout
69482026-03-24T21:03:10.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
69492026-03-24T21:03:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
69502026-03-24T21:03:10.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69512026-03-24T21:03:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
69522026-03-24T21:03:10.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
69532026-03-24T21:03:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
69542026-03-24T21:03:10.538ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
69552026-03-24T21:03:10.538ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.598368ms iteration = 74 status = {"config_updated":false}
69562026-03-24T21:03:15.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
69572026-03-24T21:03:15.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 75 reason = Timeout
69582026-03-24T21:03:15.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69592026-03-24T21:03:15.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
69602026-03-24T21:03:15.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
69612026-03-24T21:03:15.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
69622026-03-24T21:03:15.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69632026-03-24T21:03:15.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69642026-03-24T21:03:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
69652026-03-24T21:03:15.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69662026-03-24T21:03:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
69672026-03-24T21:03:15.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
69682026-03-24T21:03:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
69692026-03-24T21:03:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
69702026-03-24T21:03:15.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
69712026-03-24T21:03:15.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.366358ms iteration = 75 status = {"config_updated":false}
69722026-03-24T21:03:15.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69732026-03-24T21:03:15.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
69742026-03-24T21:03:15.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69752026-03-24T21:03:15.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
69762026-03-24T21:03:15.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
69772026-03-24T21:03:15.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
69782026-03-24T21:03:15.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
69792026-03-24T21:03:15.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69802026-03-24T21:03:15.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69812026-03-24T21:03:15.556ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
69822026-03-24T21:03:15.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
69832026-03-24T21:03:15.558ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
69842026-03-24T21:03:15.558ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.351711ms iteration = 38 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
69852026-03-24T21:03:15.599ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 44.814908ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
69862026-03-24T21:03:18.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 426 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = cf36f01c-8b54-4495-b641-3ab5ccf1fb74 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
69872026-03-24T21:03:20.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 35 reason = Timeout
69882026-03-24T21:03:20.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 76 reason = Timeout
69892026-03-24T21:03:20.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
69902026-03-24T21:03:20.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
69912026-03-24T21:03:20.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
69922026-03-24T21:03:20.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
69932026-03-24T21:03:20.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69942026-03-24T21:03:20.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69952026-03-24T21:03:20.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
69962026-03-24T21:03:20.537ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
69972026-03-24T21:03:20.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
69982026-03-24T21:03:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
69992026-03-24T21:03:20.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
70002026-03-24T21:03:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
70012026-03-24T21:03:20.539ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = bfcfe115-13a5-4646-a408-f5f6ef741c06 original_time_started = 2026-03-24 21:03:06.651617 UTC
70022026-03-24T21:03:20.539ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.615616ms iteration = 35 status = {"Loaded":{"collection_id":"bfcfe115-13a5-4646-a408-f5f6ef741c06","time_loaded":"2026-03-24T21:03:20.532282567Z","time_started":"2026-03-24T21:03:06.651617Z"}}
70032026-03-24T21:03:20.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
70042026-03-24T21:03:20.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.690857ms iteration = 76 status = {"config_updated":false}
70052026-03-24T21:03:20.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 26 reason = Timeout
70062026-03-24T21:03:20.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 26 reason = Timeout
70072026-03-24T21:03:20.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
70082026-03-24T21:03:20.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
70092026-03-24T21:03:20.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
70102026-03-24T21:03:20.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
70112026-03-24T21:03:20.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
70122026-03-24T21:03:20.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
70132026-03-24T21:03:20.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70142026-03-24T21:03:20.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70152026-03-24T21:03:20.579ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
70162026-03-24T21:03:20.579ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
70172026-03-24T21:03:20.579ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
70182026-03-24T21:03:20.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
70192026-03-24T21:03:20.580ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.376055ms iteration = 26 status = "NoSitrep"
70202026-03-24T21:03:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
70212026-03-24T21:03:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
70222026-03-24T21:03:20.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
70232026-03-24T21:03:20.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
70242026-03-24T21:03:20.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70252026-03-24T21:03:20.586ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
70262026-03-24T21:03:20.589ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
70272026-03-24T21:03:20.589ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
70282026-03-24T21:03:20.589ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 14.665446ms iteration = 26 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
70292026-03-24T21:03:25.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
70302026-03-24T21:03:25.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 77 reason = Timeout
70312026-03-24T21:03:25.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70322026-03-24T21:03:25.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
70332026-03-24T21:03:25.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
70342026-03-24T21:03:25.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
70352026-03-24T21:03:25.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70362026-03-24T21:03:25.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70372026-03-24T21:03:25.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
70382026-03-24T21:03:25.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
70392026-03-24T21:03:25.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
70402026-03-24T21:03:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
70412026-03-24T21:03:25.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70422026-03-24T21:03:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
70432026-03-24T21:03:25.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
70442026-03-24T21:03:25.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.824294ms iteration = 77 status = {"config_updated":false}
70452026-03-24T21:03:25.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70462026-03-24T21:03:25.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
70472026-03-24T21:03:25.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70482026-03-24T21:03:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
70492026-03-24T21:03:25.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70502026-03-24T21:03:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
70512026-03-24T21:03:25.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
70522026-03-24T21:03:25.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
70532026-03-24T21:03:25.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
70542026-03-24T21:03:25.556ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
70552026-03-24T21:03:25.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
70562026-03-24T21:03:25.556ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
70572026-03-24T21:03:25.557ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.82667ms iteration = 39 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
70582026-03-24T21:03:25.558ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.37948ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
70592026-03-24T21:03:28.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 500 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 62dec289-663e-4291-a2a6-4d3c61eac318 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
70602026-03-24T21:03:30.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 78 reason = Timeout
70612026-03-24T21:03:30.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
70622026-03-24T21:03:30.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
70632026-03-24T21:03:30.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70642026-03-24T21:03:30.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
70652026-03-24T21:03:30.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
70662026-03-24T21:03:30.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
70672026-03-24T21:03:30.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
70682026-03-24T21:03:30.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.027879ms iteration = 78 status = {"config_updated":false}
70692026-03-24T21:03:35.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 14 reason = Timeout
70702026-03-24T21:03:35.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
70712026-03-24T21:03:35.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
70722026-03-24T21:03:35.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
70732026-03-24T21:03:35.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
70742026-03-24T21:03:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
70752026-03-24T21:03:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
70762026-03-24T21:03:35.523ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1201, tv_nsec: 737283046 } })
70772026-03-24T21:03:35.523ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
70782026-03-24T21:03:35.524ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1201, tv_nsec: 738195801 } })
70792026-03-24T21:03:35.525ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
70802026-03-24T21:03:35.525ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
70812026-03-24T21:03:35.525ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.240101ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
70822026-03-24T21:03:35.526ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "38e54861-0b6d-4726-a2d6-671f6734215e", "content-length": "26", "date": "Tue, 24 Mar 2026 21:03:35 GMT"} })
70832026-03-24T21:03:35.526ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
70842026-03-24T21:03:35.526ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
70852026-03-24T21:03:35.527ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1201, tv_nsec: 741073438 } })
70862026-03-24T21:03:35.528ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 8.278193ms iteration = 14 status = {}
70872026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
70882026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
70892026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
70902026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
70912026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
70922026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 36 reason = Timeout
70932026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
70942026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 79 reason = Timeout
70952026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
70962026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
70972026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
70982026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
70992026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
71002026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
71012026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
71022026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71032026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
71042026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71052026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
71062026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71072026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71082026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
71092026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
71102026-03-24T21:03:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71112026-03-24T21:03:35.537ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71122026-03-24T21:03:35.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71132026-03-24T21:03:35.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71142026-03-24T21:03:35.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71152026-03-24T21:03:35.538ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71162026-03-24T21:03:35.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71172026-03-24T21:03:35.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71182026-03-24T21:03:35.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
71192026-03-24T21:03:35.539ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
71202026-03-24T21:03:35.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71212026-03-24T21:03:35.541ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71222026-03-24T21:03:35.542ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
71232026-03-24T21:03:35.542ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71242026-03-24T21:03:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71252026-03-24T21:03:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
71262026-03-24T21:03:35.543ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71272026-03-24T21:03:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71282026-03-24T21:03:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
71292026-03-24T21:03:35.544ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
71302026-03-24T21:03:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
71312026-03-24T21:03:35.545ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
71322026-03-24T21:03:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
71332026-03-24T21:03:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
71342026-03-24T21:03:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
71352026-03-24T21:03:35.545ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
71362026-03-24T21:03:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
71372026-03-24T21:03:35.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71382026-03-24T21:03:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
71392026-03-24T21:03:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
71402026-03-24T21:03:35.546ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71412026-03-24T21:03:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71422026-03-24T21:03:35.546ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
71432026-03-24T21:03:35.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
71442026-03-24T21:03:35.546ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71452026-03-24T21:03:35.546ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
71462026-03-24T21:03:35.546ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 13.823853ms iteration = 79 status = {"config_updated":false}
71472026-03-24T21:03:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
71482026-03-24T21:03:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
71492026-03-24T21:03:35.547ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
71502026-03-24T21:03:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
71512026-03-24T21:03:35.547ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = bfcfe115-13a5-4646-a408-f5f6ef741c06 original_time_started = 2026-03-24 21:03:06.651617 UTC
71522026-03-24T21:03:35.547ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 14.513342ms iteration = 36 status = {"Loaded":{"collection_id":"bfcfe115-13a5-4646-a408-f5f6ef741c06","time_loaded":"2026-03-24T21:03:35.533071090Z","time_started":"2026-03-24T21:03:06.651617Z"}}
71532026-03-24T21:03:35.548ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71542026-03-24T21:03:35.548ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1201, tv_nsec: 762662070 } })
71552026-03-24T21:03:35.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
71562026-03-24T21:03:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
71572026-03-24T21:03:35.548ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
71582026-03-24T21:03:35.549ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
71592026-03-24T21:03:35.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
71602026-03-24T21:03:35.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
71612026-03-24T21:03:35.549ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1201, tv_nsec: 763508983 } })
71622026-03-24T21:03:35.549ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 16.960347ms iteration = 14 status = 0
71632026-03-24T21:03:35.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71642026-03-24T21:03:35.551ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
71652026-03-24T21:03:35.551ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
71662026-03-24T21:03:35.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
71672026-03-24T21:03:35.551ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71682026-03-24T21:03:35.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71692026-03-24T21:03:35.551ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f478385-4a4e-49a9-b936-e722324895c7", "content-length": "26", "date": "Tue, 24 Mar 2026 21:03:35 GMT"} })
71702026-03-24T21:03:35.551ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
71712026-03-24T21:03:35.551ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
71722026-03-24T21:03:35.553ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1201, tv_nsec: 767128291 } })
71732026-03-24T21:03:35.553ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
71742026-03-24T21:03:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
71752026-03-24T21:03:35.553ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
71762026-03-24T21:03:35.553ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 20.501593ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
71772026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
71782026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
71792026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
71802026-03-24T21:03:35.554ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
71812026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
71822026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
71832026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
71842026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
71852026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
71862026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
71872026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
71882026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
71892026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
71902026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
71912026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
71922026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
71932026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
71942026-03-24T21:03:35.554ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
71952026-03-24T21:03:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
71962026-03-24T21:03:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
71972026-03-24T21:03:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
71982026-03-24T21:03:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
71992026-03-24T21:03:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
72002026-03-24T21:03:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
72012026-03-24T21:03:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
72022026-03-24T21:03:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
72032026-03-24T21:03:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
72042026-03-24T21:03:35.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
72052026-03-24T21:03:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
72062026-03-24T21:03:35.555ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1201, tv_nsec: 769915126 } })
72072026-03-24T21:03:35.556ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.006084ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
72082026-03-24T21:03:35.556ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72092026-03-24T21:03:35.557ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 14 reason = Timeout
72102026-03-24T21:03:35.557ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
72112026-03-24T21:03:35.557ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72122026-03-24T21:03:35.557ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
72132026-03-24T21:03:35.557ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72142026-03-24T21:03:35.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
72152026-03-24T21:03:35.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
72162026-03-24T21:03:35.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72172026-03-24T21:03:35.558ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
72182026-03-24T21:03:35.558ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
72192026-03-24T21:03:35.558ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
72202026-03-24T21:03:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
72212026-03-24T21:03:35.559ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.371558ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
72222026-03-24T21:03:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
72232026-03-24T21:03:35.559ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
72242026-03-24T21:03:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
72252026-03-24T21:03:35.560ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
72262026-03-24T21:03:35.560ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72272026-03-24T21:03:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72282026-03-24T21:03:35.561ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
72292026-03-24T21:03:35.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72302026-03-24T21:03:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72312026-03-24T21:03:35.563ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72322026-03-24T21:03:35.563ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72332026-03-24T21:03:35.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
72342026-03-24T21:03:35.565ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 8.164781ms iteration = 14 status = {"errors":[],"requests_created_ok":[]}
72352026-03-24T21:03:35.566ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
72362026-03-24T21:03:35.566ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72372026-03-24T21:03:35.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72382026-03-24T21:03:35.569ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72392026-03-24T21:03:35.571ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
72402026-03-24T21:03:35.571ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
72412026-03-24T21:03:35.572ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a821e5b-f504-4a76-8d28-5f33e2932970", "content-length": "372", "date": "Tue, 24 Mar 2026 21:03:35 GMT"} })
72422026-03-24T21:03:35.572ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
72432026-03-24T21:03:35.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
72442026-03-24T21:03:35.575ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 14 reason = Timeout
72452026-03-24T21:03:35.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 27 reason = Timeout
72462026-03-24T21:03:35.576ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 20.45\u{b5}s iteration = 14 status = {"disabled":true,"errors":[],"sps":[]}
72472026-03-24T21:03:35.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
72482026-03-24T21:03:35.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 27 reason = Timeout
72492026-03-24T21:03:35.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
72502026-03-24T21:03:35.576ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
72512026-03-24T21:03:35.576ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72522026-03-24T21:03:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
72532026-03-24T21:03:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
72542026-03-24T21:03:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
72552026-03-24T21:03:35.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
72562026-03-24T21:03:35.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
72572026-03-24T21:03:35.578ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
72582026-03-24T21:03:35.578ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 23.867983ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
72592026-03-24T21:03:35.580ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72602026-03-24T21:03:35.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72612026-03-24T21:03:35.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72622026-03-24T21:03:35.583ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
72632026-03-24T21:03:35.583ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
72642026-03-24T21:03:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
72652026-03-24T21:03:35.585ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
72662026-03-24T21:03:35.585ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
72672026-03-24T21:03:35.585ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
72682026-03-24T21:03:35.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
72692026-03-24T21:03:35.587ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
72702026-03-24T21:03:35.587ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 53.980325ms iteration = 40 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
72712026-03-24T21:03:35.588ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
72722026-03-24T21:03:35.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 12.504157ms iteration = 27 status = "NoSitrep"
72732026-03-24T21:03:35.588ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "053d3153-15c0-4466-9c28-d4d46b95f023", "content-length": "691", "date": "Tue, 24 Mar 2026 21:03:35 GMT"} })
72742026-03-24T21:03:35.588ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
72752026-03-24T21:03:35.588ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
72762026-03-24T21:03:35.588ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
72772026-03-24T21:03:35.588ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 55.873397ms iteration = 14 status = {}
72782026-03-24T21:03:35.589ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
72792026-03-24T21:03:35.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
72802026-03-24T21:03:35.590ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
72812026-03-24T21:03:35.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
72822026-03-24T21:03:35.591ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
72832026-03-24T21:03:35.591ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
72842026-03-24T21:03:35.592ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72852026-03-24T21:03:35.592ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
72862026-03-24T21:03:35.592ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
72872026-03-24T21:03:35.592ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
72882026-03-24T21:03:35.592ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
72892026-03-24T21:03:35.592ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
72902026-03-24T21:03:35.592ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
72912026-03-24T21:03:35.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
72922026-03-24T21:03:35.594ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
72932026-03-24T21:03:35.596ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
72942026-03-24T21:03:35.596ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
72952026-03-24T21:03:35.600ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
72962026-03-24T21:03:35.600ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
72972026-03-24T21:03:35.600ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 24.399658ms iteration = 27 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
72982026-03-24T21:03:35.620ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1201, tv_nsec: 834503598 } })
72992026-03-24T21:03:35.621ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
73002026-03-24T21:03:35.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
73012026-03-24T21:03:35.622ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
73022026-03-24T21:03:35.622ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 101.718563ms iteration = 14 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
73032026-03-24T21:03:35.624ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 77.987013ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
73042026-03-24T21:03:35.630ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
73052026-03-24T21:03:35.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
73062026-03-24T21:03:35.633ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 78.512827ms iteration = 14 status = {"errors":[],"finish_invoked_ok":[]}
73072026-03-24T21:03:35.633ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
73082026-03-24T21:03:35.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
73092026-03-24T21:03:35.637ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 83.001707ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
73102026-03-24T21:03:35.638ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 104.863397ms iteration = 24 status = {"physical_disks_added":0}
73112026-03-24T21:03:35.643ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
73122026-03-24T21:03:35.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
73132026-03-24T21:03:35.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 90.899971ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
73142026-03-24T21:03:35.647ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
73152026-03-24T21:03:35.647ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
73162026-03-24T21:03:35.647ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
73172026-03-24T21:03:35.647ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
73182026-03-24T21:03:35.647ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
73192026-03-24T21:03:35.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
73202026-03-24T21:03:35.648ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "30242ee3-0e41-423d-ad1e-f3ac3f5115db", "date": "Tue, 24 Mar 2026 21:03:35 GMT"} })
73212026-03-24T21:03:35.648ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73222026-03-24T21:03:35.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73232026-03-24T21:03:35.650ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73242026-03-24T21:03:35.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73252026-03-24T21:03:35.653ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
73262026-03-24T21:03:35.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
73272026-03-24T21:03:35.656ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
73282026-03-24T21:03:35.656ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
73292026-03-24T21:03:35.656ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "29a7a912-cf68-4c15-824b-e82d69976ded", "date": "Tue, 24 Mar 2026 21:03:35 GMT"} })
73302026-03-24T21:03:35.656ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 102.304169ms iteration = 21 status = {}
73312026-03-24T21:03:35.696ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
73322026-03-24T21:03:35.696ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
73332026-03-24T21:03:35.696ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 141.778394ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
73342026-03-24T21:03:35.746ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c7c542f2-1936-412d-8362-2bb410a70794", "date": "Tue, 24 Mar 2026 21:03:35 GMT"} })
73352026-03-24T21:03:35.746ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
73362026-03-24T21:03:35.746ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73372026-03-24T21:03:35.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73382026-03-24T21:03:35.749ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73392026-03-24T21:03:35.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73402026-03-24T21:03:35.753ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73412026-03-24T21:03:35.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73422026-03-24T21:03:35.756ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
73432026-03-24T21:03:35.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
73442026-03-24T21:03:35.758ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 225.784628ms iteration = 14 status = {}
73452026-03-24T21:03:38.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 420 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 60d68436-e401-47a3-8394-97571e24ea6d response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
73462026-03-24T21:03:40.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 80 reason = Timeout
73472026-03-24T21:03:40.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
73482026-03-24T21:03:40.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
73492026-03-24T21:03:40.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73502026-03-24T21:03:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
73512026-03-24T21:03:40.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
73522026-03-24T21:03:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
73532026-03-24T21:03:40.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
73542026-03-24T21:03:40.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.934357ms iteration = 80 status = {"config_updated":false}
73552026-03-24T21:03:45.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
73562026-03-24T21:03:45.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 81 reason = Timeout
73572026-03-24T21:03:45.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
73582026-03-24T21:03:45.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73592026-03-24T21:03:45.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
73602026-03-24T21:03:45.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
73612026-03-24T21:03:45.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73622026-03-24T21:03:45.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73632026-03-24T21:03:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
73642026-03-24T21:03:45.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73652026-03-24T21:03:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
73662026-03-24T21:03:45.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
73672026-03-24T21:03:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
73682026-03-24T21:03:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
73692026-03-24T21:03:45.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
73702026-03-24T21:03:45.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.205174ms iteration = 81 status = {"config_updated":false}
73712026-03-24T21:03:45.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73722026-03-24T21:03:45.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
73732026-03-24T21:03:45.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73742026-03-24T21:03:45.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
73752026-03-24T21:03:45.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73762026-03-24T21:03:45.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
73772026-03-24T21:03:45.555ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
73782026-03-24T21:03:45.555ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73792026-03-24T21:03:45.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73802026-03-24T21:03:45.556ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
73812026-03-24T21:03:45.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
73822026-03-24T21:03:45.559ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
73832026-03-24T21:03:45.559ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.678439ms iteration = 41 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
73842026-03-24T21:03:45.597ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.27469ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
73852026-03-24T21:03:48.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 496 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = e731da39-8d5a-43b9-898c-7b9c12362a4e response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
73862026-03-24T21:03:50.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 37 reason = Timeout
73872026-03-24T21:03:50.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 82 reason = Timeout
73882026-03-24T21:03:50.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
73892026-03-24T21:03:50.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
73902026-03-24T21:03:50.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
73912026-03-24T21:03:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
73922026-03-24T21:03:50.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73932026-03-24T21:03:50.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73942026-03-24T21:03:50.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
73952026-03-24T21:03:50.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
73962026-03-24T21:03:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
73972026-03-24T21:03:50.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
73982026-03-24T21:03:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
73992026-03-24T21:03:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
74002026-03-24T21:03:50.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
74012026-03-24T21:03:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.610896ms iteration = 82 status = {"config_updated":false}
74022026-03-24T21:03:50.540ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = bfcfe115-13a5-4646-a408-f5f6ef741c06 original_time_started = 2026-03-24 21:03:06.651617 UTC
74032026-03-24T21:03:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.672706ms iteration = 37 status = {"Loaded":{"collection_id":"bfcfe115-13a5-4646-a408-f5f6ef741c06","time_loaded":"2026-03-24T21:03:50.532452642Z","time_started":"2026-03-24T21:03:06.651617Z"}}
74042026-03-24T21:03:50.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 28 reason = Timeout
74052026-03-24T21:03:50.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 28 reason = Timeout
74062026-03-24T21:03:50.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
74072026-03-24T21:03:50.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
74082026-03-24T21:03:50.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
74092026-03-24T21:03:50.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
74102026-03-24T21:03:50.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
74112026-03-24T21:03:50.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
74122026-03-24T21:03:50.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74132026-03-24T21:03:50.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74142026-03-24T21:03:50.579ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
74152026-03-24T21:03:50.579ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
74162026-03-24T21:03:50.579ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
74172026-03-24T21:03:50.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
74182026-03-24T21:03:50.581ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.5769ms iteration = 28 status = "NoSitrep"
74192026-03-24T21:03:50.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
74202026-03-24T21:03:50.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
74212026-03-24T21:03:50.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
74222026-03-24T21:03:50.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
74232026-03-24T21:03:50.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74242026-03-24T21:03:50.585ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
74252026-03-24T21:03:50.588ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
74262026-03-24T21:03:50.588ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
74272026-03-24T21:03:50.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.091026ms iteration = 28 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
74282026-03-24T21:03:55.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
74292026-03-24T21:03:55.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 83 reason = Timeout
74302026-03-24T21:03:55.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74312026-03-24T21:03:55.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74322026-03-24T21:03:55.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74332026-03-24T21:03:55.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74342026-03-24T21:03:55.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74352026-03-24T21:03:55.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74362026-03-24T21:03:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
74372026-03-24T21:03:55.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74382026-03-24T21:03:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
74392026-03-24T21:03:55.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74402026-03-24T21:03:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74412026-03-24T21:03:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74422026-03-24T21:03:55.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74432026-03-24T21:03:55.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74442026-03-24T21:03:55.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
74452026-03-24T21:03:55.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.530553ms iteration = 83 status = {"config_updated":false}
74462026-03-24T21:03:55.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74472026-03-24T21:03:55.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
74482026-03-24T21:03:55.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74492026-03-24T21:03:55.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74502026-03-24T21:03:55.554ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
74512026-03-24T21:03:55.554ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74522026-03-24T21:03:55.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74532026-03-24T21:03:55.555ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74542026-03-24T21:03:55.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74552026-03-24T21:03:55.556ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
74562026-03-24T21:03:55.556ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.477073ms iteration = 42 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
74572026-03-24T21:03:55.557ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.135635ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74582026-03-24T21:03:58.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 478 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 6ca8b6b8-5098-4f4b-b505-acc36869e07f response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
74592026-03-24T21:04:00.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 84 reason = Timeout
74602026-03-24T21:04:00.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74612026-03-24T21:04:00.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74622026-03-24T21:04:00.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74632026-03-24T21:04:00.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
74642026-03-24T21:04:00.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74652026-03-24T21:04:00.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74662026-03-24T21:04:00.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
74672026-03-24T21:04:00.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.989769ms iteration = 84 status = {"config_updated":false}
74682026-03-24T21:04:05.515ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
74692026-03-24T21:04:05.515ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
74702026-03-24T21:04:05.515ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
74712026-03-24T21:04:05.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
74722026-03-24T21:04:05.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
74732026-03-24T21:04:05.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
74742026-03-24T21:04:05.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74752026-03-24T21:04:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
74762026-03-24T21:04:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
74772026-03-24T21:04:05.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
74782026-03-24T21:04:05.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
74792026-03-24T21:04:05.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
74802026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
74812026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
74822026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 15 reason = Timeout
74832026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
74842026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
74852026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
74862026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
74872026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
74882026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
74892026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
74902026-03-24T21:04:05.521ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2026-03-24 20:54:05.521110278 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
74912026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
74922026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
74932026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
74942026-03-24T21:04:05.521ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
74952026-03-24T21:04:05.521ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = ExternalNetworking step id = Ensure
74962026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
74972026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
74982026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
74992026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
75002026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
75012026-03-24T21:04:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("93b6e989-1f7d-4884-aec9-ecbc076ed6c4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 735811446 } })
75022026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
75032026-03-24T21:04:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75042026-03-24T21:04:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("27f3a9c5-67c7-41c5-994d-9db513cda557.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2f7ea2ed-fd45-4cd7-9fb1-f48119541118.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 735861908 } })
75052026-03-24T21:04:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
75062026-03-24T21:04:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.50333ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
75072026-03-24T21:04:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
75082026-03-24T21:04:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.448339ms iteration = 8 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
75092026-03-24T21:04:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
75102026-03-24T21:04:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
75112026-03-24T21:04:05.523ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
75122026-03-24T21:04:05.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
75132026-03-24T21:04:05.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
75142026-03-24T21:04:05.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
75152026-03-24T21:04:05.524ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 738553430 } })
75162026-03-24T21:04:05.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
75172026-03-24T21:04:05.524ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75182026-03-24T21:04:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "fc0ac20c-5b68-462e-af70-279dd78b227e", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
75192026-03-24T21:04:05.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
75202026-03-24T21:04:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "5f6c6e52-d5c1-465e-8e3a-978da2a33bac", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
75212026-03-24T21:04:05.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
75222026-03-24T21:04:05.524ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
75232026-03-24T21:04:05.524ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 3.859034ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
75242026-03-24T21:04:05.525ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2026-03-24 20:54:05.521110278 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
75252026-03-24T21:04:05.525ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.183373ms iteration = 8 status = {"expiration":"2026-03-24T20:54:05.521110278Z","pruned":[]}
75262026-03-24T21:04:05.525ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 739326311 } })
75272026-03-24T21:04:05.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
75282026-03-24T21:04:05.526ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
75292026-03-24T21:04:05.526ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 740254286 } })
75302026-03-24T21:04:05.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75312026-03-24T21:04:05.526ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
75322026-03-24T21:04:05.526ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
75332026-03-24T21:04:05.526ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
75342026-03-24T21:04:05.526ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
75352026-03-24T21:04:05.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75362026-03-24T21:04:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "9e2e8eed-e5af-4e6d-82af-7165280bdab2", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
75372026-03-24T21:04:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
75382026-03-24T21:04:05.526ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75392026-03-24T21:04:05.527ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "1fc55fb8-9848-4430-94a2-1e37cbb2a741", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
75402026-03-24T21:04:05.527ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
75412026-03-24T21:04:05.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
75422026-03-24T21:04:05.527ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
75432026-03-24T21:04:05.527ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "19189e12-f1a1-4075-b444-48cd7ca4b5a9", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
75442026-03-24T21:04:05.527ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
75452026-03-24T21:04:05.527ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 6.608068ms iteration = 15 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
75462026-03-24T21:04:05.527ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
75472026-03-24T21:04:05.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 6.603788ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
75482026-03-24T21:04:05.528ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.081511ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
75492026-03-24T21:04:05.528ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
75502026-03-24T21:04:05.528ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
75512026-03-24T21:04:05.528ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
75522026-03-24T21:04:05.528ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
75532026-03-24T21:04:05.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
75542026-03-24T21:04:05.528ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b66e1bd4-9e41-4b86-a2a9-29c9b1da193c", "content-length": "26", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} })
75552026-03-24T21:04:05.528ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
75562026-03-24T21:04:05.528ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
75572026-03-24T21:04:05.529ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 743370590 } })
75582026-03-24T21:04:05.529ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75592026-03-24T21:04:05.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
75602026-03-24T21:04:05.531ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
75612026-03-24T21:04:05.531ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 10.696259ms iteration = 15 status = {}
75622026-03-24T21:04:05.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
75632026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
75642026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
75652026-03-24T21:04:05.532ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
75662026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
75672026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75682026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
75692026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
75702026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 20 reason = Timeout
75712026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75722026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
75732026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
75742026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
75752026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
75762026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75772026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 38 reason = Timeout
75782026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
75792026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
75802026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75812026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
75822026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 85 reason = Timeout
75832026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75842026-03-24T21:04:05.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75852026-03-24T21:04:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75862026-03-24T21:04:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
75872026-03-24T21:04:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75882026-03-24T21:04:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
75892026-03-24T21:04:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
75902026-03-24T21:04:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75912026-03-24T21:04:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75922026-03-24T21:04:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
75932026-03-24T21:04:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75942026-03-24T21:04:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
75952026-03-24T21:04:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75962026-03-24T21:04:05.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75972026-03-24T21:04:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
75982026-03-24T21:04:05.535ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75992026-03-24T21:04:05.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76002026-03-24T21:04:05.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76012026-03-24T21:04:05.535ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
76022026-03-24T21:04:05.535ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
76032026-03-24T21:04:05.535ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
76042026-03-24T21:04:05.535ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
76052026-03-24T21:04:05.535ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76062026-03-24T21:04:05.535ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76072026-03-24T21:04:05.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76082026-03-24T21:04:05.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
76092026-03-24T21:04:05.536ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76102026-03-24T21:04:05.536ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76112026-03-24T21:04:05.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
76122026-03-24T21:04:05.536ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
76132026-03-24T21:04:05.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76142026-03-24T21:04:05.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
76152026-03-24T21:04:05.537ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76162026-03-24T21:04:05.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76172026-03-24T21:04:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
76182026-03-24T21:04:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
76192026-03-24T21:04:05.542ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76202026-03-24T21:04:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
76212026-03-24T21:04:05.542ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76222026-03-24T21:04:05.542ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
76232026-03-24T21:04:05.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
76242026-03-24T21:04:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76252026-03-24T21:04:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76262026-03-24T21:04:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
76272026-03-24T21:04:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
76282026-03-24T21:04:05.543ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
76292026-03-24T21:04:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
76302026-03-24T21:04:05.543ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
76312026-03-24T21:04:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
76322026-03-24T21:04:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
76332026-03-24T21:04:05.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76342026-03-24T21:04:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
76352026-03-24T21:04:05.544ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
76362026-03-24T21:04:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
76372026-03-24T21:04:05.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76382026-03-24T21:04:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
76392026-03-24T21:04:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
76402026-03-24T21:04:05.545ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
76412026-03-24T21:04:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
76422026-03-24T21:04:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
76432026-03-24T21:04:05.546ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
76442026-03-24T21:04:05.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
76452026-03-24T21:04:05.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
76462026-03-24T21:04:05.546ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
76472026-03-24T21:04:05.546ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76482026-03-24T21:04:05.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
76492026-03-24T21:04:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
76502026-03-24T21:04:05.547ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76512026-03-24T21:04:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
76522026-03-24T21:04:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76532026-03-24T21:04:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76542026-03-24T21:04:05.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76552026-03-24T21:04:05.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
76562026-03-24T21:04:05.549ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76572026-03-24T21:04:05.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76582026-03-24T21:04:05.551ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
76592026-03-24T21:04:05.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
76602026-03-24T21:04:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
76612026-03-24T21:04:05.553ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 767198283 } })
76622026-03-24T21:04:05.557ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
76632026-03-24T21:04:05.557ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
76642026-03-24T21:04:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
76652026-03-24T21:04:05.557ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
76662026-03-24T21:04:05.557ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
76672026-03-24T21:04:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
76682026-03-24T21:04:05.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76692026-03-24T21:04:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
76702026-03-24T21:04:05.558ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
76712026-03-24T21:04:05.558ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
76722026-03-24T21:04:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
76732026-03-24T21:04:05.558ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = bfcfe115-13a5-4646-a408-f5f6ef741c06 original_time_started = 2026-03-24 21:03:06.651617 UTC
76742026-03-24T21:04:05.558ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 25.969581ms iteration = 38 status = {"Loaded":{"collection_id":"bfcfe115-13a5-4646-a408-f5f6ef741c06","time_loaded":"2026-03-24T21:04:05.532840615Z","time_started":"2026-03-24T21:03:06.651617Z"}}
76752026-03-24T21:04:05.558ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
76762026-03-24T21:04:05.558ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
76772026-03-24T21:04:05.559ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
76782026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 18 reason = Timeout
76792026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 8 reason = Timeout
76802026-03-24T21:04:05.559ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
76812026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
76822026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
76832026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 13.761\u{b5}s iteration = 18 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
76842026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 8 reason = Timeout
76852026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
76862026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
76872026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
76882026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 15 reason = Timeout
76892026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
76902026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
76912026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76922026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
76932026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
76942026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
76952026-03-24T21:04:05.559ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
76962026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76972026-03-24T21:04:05.559ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
76982026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
76992026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
77002026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
77012026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
77022026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
77032026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
77042026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
77052026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77062026-03-24T21:04:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
77072026-03-24T21:04:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
77082026-03-24T21:04:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77092026-03-24T21:04:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
77102026-03-24T21:04:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
77112026-03-24T21:04:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
77122026-03-24T21:04:05.560ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 773968375 } })
77132026-03-24T21:04:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77142026-03-24T21:04:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
77152026-03-24T21:04:05.561ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
77162026-03-24T21:04:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
77172026-03-24T21:04:05.562ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
77182026-03-24T21:04:05.562ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
77192026-03-24T21:04:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
77202026-03-24T21:04:05.563ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
77212026-03-24T21:04:05.565ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e328dedf-1406-4ed5-8832-8113a852e896", "content-length": "26", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} })
77222026-03-24T21:04:05.565ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77232026-03-24T21:04:05.565ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
77242026-03-24T21:04:05.565ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
77252026-03-24T21:04:05.565ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77262026-03-24T21:04:05.566ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
77272026-03-24T21:04:05.566ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 33.983366ms iteration = 85 status = {"config_updated":false}
77282026-03-24T21:04:05.567ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 781093277 } })
77292026-03-24T21:04:05.567ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
77302026-03-24T21:04:05.567ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77312026-03-24T21:04:05.567ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 8.425197ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
77322026-03-24T21:04:05.567ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
77332026-03-24T21:04:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
77342026-03-24T21:04:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
77352026-03-24T21:04:05.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77362026-03-24T21:04:05.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
77372026-03-24T21:04:05.568ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
77382026-03-24T21:04:05.569ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 36.202066ms iteration = 15 status = 0
77392026-03-24T21:04:05.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77402026-03-24T21:04:05.569ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
77412026-03-24T21:04:05.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
77422026-03-24T21:04:05.569ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 783915943 } })
77432026-03-24T21:04:05.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
77442026-03-24T21:04:05.570ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
77452026-03-24T21:04:05.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77462026-03-24T21:04:05.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
77472026-03-24T21:04:05.571ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77482026-03-24T21:04:05.572ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77492026-03-24T21:04:05.572ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
77502026-03-24T21:04:05.572ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77512026-03-24T21:04:05.572ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77522026-03-24T21:04:05.572ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
77532026-03-24T21:04:05.572ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 13.292198ms iteration = 8 status = {"by_rx":{},"error":null}
77542026-03-24T21:04:05.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77552026-03-24T21:04:05.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77562026-03-24T21:04:05.572ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 13.627577ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
77572026-03-24T21:04:05.573ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
77582026-03-24T21:04:05.573ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
77592026-03-24T21:04:05.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
77602026-03-24T21:04:05.575ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
77612026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 16.183027ms iteration = 15 status = {"errors":[],"requests_created_ok":[]}
77622026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 18.414417ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
77632026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
77642026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
77652026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
77662026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
77672026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): activating background_task = trust_quorum_manager iteration = 8 reason = Timeout
77682026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 15 reason = Timeout
77692026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
77702026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 29 reason = Timeout
77712026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 16.92\u{b5}s iteration = 15 status = {"disabled":true,"errors":[],"sps":[]}
77722026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
77732026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): activating background_task = attached_subnet_manager iteration = 8 reason = Timeout
77742026-03-24T21:04:05.575ZINFOnexus (BackgroundTasks): starting attached subnet manager work background_task = attached_subnet_manager file = nexus/src/app/background/tasks/attached_subnets.rs:374
77752026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 29 reason = Timeout
77762026-03-24T21:04:05.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
77772026-03-24T21:04:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
77782026-03-24T21:04:05.576ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 8 reason = Timeout
77792026-03-24T21:04:05.576ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
77802026-03-24T21:04:05.576ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
77812026-03-24T21:04:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
77822026-03-24T21:04:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
77832026-03-24T21:04:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
77842026-03-24T21:04:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
77852026-03-24T21:04:05.576ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1231, tv_nsec: 790666205 } })
77862026-03-24T21:04:05.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77872026-03-24T21:04:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
77882026-03-24T21:04:05.578ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
77892026-03-24T21:04:05.578ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
77902026-03-24T21:04:05.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
77912026-03-24T21:04:05.578ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 19.368102ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
77922026-03-24T21:04:05.578ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
77932026-03-24T21:04:05.578ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77942026-03-24T21:04:05.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77952026-03-24T21:04:05.579ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
77962026-03-24T21:04:05.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
77972026-03-24T21:04:05.579ZDEBGnexus (BackgroundTasks): roles background_task = attached_subnet_manager roles = RoleSet { roles: {} }
77982026-03-24T21:04:05.579ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
77992026-03-24T21:04:05.579ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
78002026-03-24T21:04:05.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = attached_subnet_manager resource = Database result = Ok(())
78012026-03-24T21:04:05.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78022026-03-24T21:04:05.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
78032026-03-24T21:04:05.580ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
78042026-03-24T21:04:05.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
78052026-03-24T21:04:05.581ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
78062026-03-24T21:04:05.583ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
78072026-03-24T21:04:05.583ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78082026-03-24T21:04:05.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78092026-03-24T21:04:05.585ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
78102026-03-24T21:04:05.586ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
78112026-03-24T21:04:05.586ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = SledList step id = Fetch
78122026-03-24T21:04:05.587ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78132026-03-24T21:04:05.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78142026-03-24T21:04:05.587ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Fleet result = Ok(())
78152026-03-24T21:04:05.587ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
78162026-03-24T21:04:05.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
78172026-03-24T21:04:05.588ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 29.311251ms iteration = 8 status = {"dispatched":[],"errors":[],"glob_version":"242.0.0","globs_reprocessed":{},"no_receivers":[]}
78182026-03-24T21:04:05.588ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78192026-03-24T21:04:05.589ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
78202026-03-24T21:04:05.589ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
78212026-03-24T21:04:05.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
78222026-03-24T21:04:05.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78232026-03-24T21:04:05.592ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
78242026-03-24T21:04:05.592ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
78252026-03-24T21:04:05.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
78262026-03-24T21:04:05.594ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78272026-03-24T21:04:05.596ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
78282026-03-24T21:04:05.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
78292026-03-24T21:04:05.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
78302026-03-24T21:04:05.596ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
78312026-03-24T21:04:05.596ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 50.407349ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
78322026-03-24T21:04:05.603ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
78332026-03-24T21:04:05.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
78342026-03-24T21:04:05.614ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
78352026-03-24T21:04:05.614ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 81.825057ms iteration = 43 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
78362026-03-24T21:04:05.618ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 41.500289ms iteration = 29 status = "NoSitrep"
78372026-03-24T21:04:05.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78382026-03-24T21:04:05.620ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
78392026-03-24T21:04:05.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
78402026-03-24T21:04:05.621ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
78412026-03-24T21:04:05.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78422026-03-24T21:04:05.621ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78432026-03-24T21:04:05.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78442026-03-24T21:04:05.625ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78452026-03-24T21:04:05.626ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
78462026-03-24T21:04:05.626ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
78472026-03-24T21:04:05.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
78482026-03-24T21:04:05.627ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
78492026-03-24T21:04:05.627ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
78502026-03-24T21:04:05.628ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
78512026-03-24T21:04:05.628ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
78522026-03-24T21:04:05.628ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
78532026-03-24T21:04:05.628ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
78542026-03-24T21:04:05.628ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
78552026-03-24T21:04:05.628ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78562026-03-24T21:04:05.628ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
78572026-03-24T21:04:05.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78582026-03-24T21:04:05.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78592026-03-24T21:04:05.632ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
78602026-03-24T21:04:05.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78612026-03-24T21:04:05.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78622026-03-24T21:04:05.632ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
78632026-03-24T21:04:05.632ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
78642026-03-24T21:04:05.637ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = DeployNexusRecords step id = Ensure
78652026-03-24T21:04:05.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78662026-03-24T21:04:05.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78672026-03-24T21:04:05.638ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
78682026-03-24T21:04:05.638ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
78692026-03-24T21:04:05.638ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 62.924627ms iteration = 29 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
78702026-03-24T21:04:05.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78712026-03-24T21:04:05.643ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
78722026-03-24T21:04:05.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78732026-03-24T21:04:05.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78742026-03-24T21:04:05.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
78752026-03-24T21:04:05.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
78762026-03-24T21:04:05.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78772026-03-24T21:04:05.649ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
78782026-03-24T21:04:05.649ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
78792026-03-24T21:04:05.649ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 91.466426ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
78802026-03-24T21:04:05.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
78812026-03-24T21:04:05.655ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
78822026-03-24T21:04:05.655ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
78832026-03-24T21:04:05.655ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 122.289877ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
78842026-03-24T21:04:05.669ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 110.670714ms iteration = 15 status = {"errors":[],"finish_invoked_ok":[]}
78852026-03-24T21:04:05.673ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
78862026-03-24T21:04:05.673ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
78872026-03-24T21:04:05.673ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
78882026-03-24T21:04:05.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
78892026-03-24T21:04:05.674ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
78902026-03-24T21:04:05.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
78912026-03-24T21:04:05.679ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2026-03-24T20:56:59.957750Z, time_modified: 2026-03-24T20:56:59.957750Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }) }
78922026-03-24T21:04:05.679ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 157.961439ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]}},"default_endpoint":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
78932026-03-24T21:04:05.679ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
78942026-03-24T21:04:05.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
78952026-03-24T21:04:05.681ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
78962026-03-24T21:04:05.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
78972026-03-24T21:04:05.681ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
78982026-03-24T21:04:05.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
78992026-03-24T21:04:05.683ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79002026-03-24T21:04:05.684ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
79012026-03-24T21:04:05.684ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 125.175396ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
79022026-03-24T21:04:05.684ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79032026-03-24T21:04:05.686ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
79042026-03-24T21:04:05.686ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
79052026-03-24T21:04:05.686ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79062026-03-24T21:04:05.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
79072026-03-24T21:04:05.686ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
79082026-03-24T21:04:05.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
79092026-03-24T21:04:05.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
79102026-03-24T21:04:05.687ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
79112026-03-24T21:04:05.687ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
79122026-03-24T21:04:05.687ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
79132026-03-24T21:04:05.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
79142026-03-24T21:04:05.689ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
79152026-03-24T21:04:05.689ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
79162026-03-24T21:04:05.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
79172026-03-24T21:04:05.689ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79182026-03-24T21:04:05.691ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
79192026-03-24T21:04:05.691ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 158.514924ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
79202026-03-24T21:04:05.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
79212026-03-24T21:04:05.695ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79222026-03-24T21:04:05.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
79232026-03-24T21:04:05.697ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79242026-03-24T21:04:05.700ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
79252026-03-24T21:04:05.700ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
79262026-03-24T21:04:05.700ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2026-03-24 20:57:05.047625 UTC dns_group = internal generation = 1
79272026-03-24T21:04:05.700ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 184.68961ms iteration = 8 status = {"generation":1}
79282026-03-24T21:04:05.701ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
79292026-03-24T21:04:05.701ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
79302026-03-24T21:04:05.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
79312026-03-24T21:04:05.708ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
79322026-03-24T21:04:05.709ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79332026-03-24T21:04:05.710ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "46859a52-dacf-41f1-b51b-d9973f1bd8fa", "content-length": "691", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} })
79342026-03-24T21:04:05.710ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
79352026-03-24T21:04:05.710ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
79362026-03-24T21:04:05.710ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
79372026-03-24T21:04:05.710ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 177.496197ms iteration = 15 status = {}
79382026-03-24T21:04:05.712ZINFOnexus (BackgroundTasks): Loaded 0 active trust quorum configurations from database background_task = trust_quorum_manager bgtask = TrustQuorumManager file = nexus/src/app/background/tasks/trust_quorum.rs:83
79392026-03-24T21:04:05.712ZDEBGnexus (BackgroundTasks): activation complete background_task = trust_quorum_manager elapsed = 136.187032ms iteration = 8 status = {"PerRackStatus":{"errors":[],"statuses":[]}}
79402026-03-24T21:04:05.713ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
79412026-03-24T21:04:05.713ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79422026-03-24T21:04:05.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79432026-03-24T21:04:05.720ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
79442026-03-24T21:04:05.720ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
79452026-03-24T21:04:05.720ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9534e89-7d9f-46e4-83f6-8d0b1da8f9b5", "content-length": "372", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} })
79462026-03-24T21:04:05.720ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79472026-03-24T21:04:05.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79482026-03-24T21:04:05.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79492026-03-24T21:04:05.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79502026-03-24T21:04:05.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79512026-03-24T21:04:05.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79522026-03-24T21:04:05.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79532026-03-24T21:04:05.741ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
79542026-03-24T21:04:05.741ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
79552026-03-24T21:04:05.741ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "accbadb6-40b0-4cc5-adda-d70e3e116939", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} })
79562026-03-24T21:04:05.741ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79572026-03-24T21:04:05.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79582026-03-24T21:04:05.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
79592026-03-24T21:04:05.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79602026-03-24T21:04:05.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79612026-03-24T21:04:05.744ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79622026-03-24T21:04:05.745ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = SledAgent step id = Ensure
79632026-03-24T21:04:05.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79642026-03-24T21:04:05.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79652026-03-24T21:04:05.746ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
79662026-03-24T21:04:05.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79672026-03-24T21:04:05.747ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
79682026-03-24T21:04:05.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
79692026-03-24T21:04:05.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79702026-03-24T21:04:05.749ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "5286a7c2-bec6-4a9d-bf35-96164802876e", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} }) sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
79712026-03-24T21:04:05.749ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = FirewallRules step id = Ensure
79722026-03-24T21:04:05.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
79732026-03-24T21:04:05.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
79742026-03-24T21:04:05.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
79752026-03-24T21:04:05.752ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 192.794539ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
79762026-03-24T21:04:05.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
79772026-03-24T21:04:05.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
79782026-03-24T21:04:05.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79792026-03-24T21:04:05.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79802026-03-24T21:04:05.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
79812026-03-24T21:04:05.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79822026-03-24T21:04:05.756ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
79832026-03-24T21:04:05.757ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
79842026-03-24T21:04:05.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
79852026-03-24T21:04:05.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
79862026-03-24T21:04:05.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79872026-03-24T21:04:05.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
79882026-03-24T21:04:05.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
79892026-03-24T21:04:05.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
79902026-03-24T21:04:05.763ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79912026-03-24T21:04:05.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
79922026-03-24T21:04:05.765ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "12bf4eb9-6133-422a-ae98-c2f7e5d853e5", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} })
79932026-03-24T21:04:05.765ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 205.857342ms iteration = 22 status = {}
79942026-03-24T21:04:05.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
79952026-03-24T21:04:05.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
79962026-03-24T21:04:05.767ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
79972026-03-24T21:04:05.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
79982026-03-24T21:04:05.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
79992026-03-24T21:04:05.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80002026-03-24T21:04:05.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80012026-03-24T21:04:05.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80022026-03-24T21:04:05.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80032026-03-24T21:04:05.770ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
80042026-03-24T21:04:05.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80052026-03-24T21:04:05.770ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
80062026-03-24T21:04:05.770ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2026-03-24 20:57:07.125641 UTC dns_group = external generation = 2
80072026-03-24T21:04:05.770ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 249.159738ms iteration = 8 status = {"generation":2}
80082026-03-24T21:04:05.775ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
80092026-03-24T21:04:05.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80102026-03-24T21:04:05.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80112026-03-24T21:04:05.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
80122026-03-24T21:04:05.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80132026-03-24T21:04:05.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80142026-03-24T21:04:05.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80152026-03-24T21:04:05.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80162026-03-24T21:04:05.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80172026-03-24T21:04:05.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80182026-03-24T21:04:05.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80192026-03-24T21:04:05.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80202026-03-24T21:04:05.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80212026-03-24T21:04:05.781ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
80222026-03-24T21:04:05.781ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
80232026-03-24T21:04:05.781ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
80242026-03-24T21:04:05.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
80252026-03-24T21:04:05.784ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
80262026-03-24T21:04:05.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80272026-03-24T21:04:05.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80282026-03-24T21:04:05.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80292026-03-24T21:04:05.788ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
80302026-03-24T21:04:05.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
80312026-03-24T21:04:05.790ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
80322026-03-24T21:04:05.790ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
80332026-03-24T21:04:05.791ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 231.730898ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
80342026-03-24T21:04:05.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80352026-03-24T21:04:05.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80362026-03-24T21:04:05.793ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80372026-03-24T21:04:05.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80382026-03-24T21:04:05.794ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
80392026-03-24T21:04:05.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80402026-03-24T21:04:05.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80412026-03-24T21:04:05.796ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
80422026-03-24T21:04:05.796ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
80432026-03-24T21:04:05.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
80442026-03-24T21:04:05.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80452026-03-24T21:04:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
80462026-03-24T21:04:05.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80472026-03-24T21:04:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80482026-03-24T21:04:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80492026-03-24T21:04:05.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80502026-03-24T21:04:05.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80512026-03-24T21:04:05.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80522026-03-24T21:04:05.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80532026-03-24T21:04:05.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80542026-03-24T21:04:05.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80552026-03-24T21:04:05.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80562026-03-24T21:04:05.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80572026-03-24T21:04:05.810ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
80582026-03-24T21:04:05.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80592026-03-24T21:04:05.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80602026-03-24T21:04:05.812ZDEBGnexus (BackgroundTasks): listed attached subnets background_task = attached_subnet_manager n_subnets = 0
80612026-03-24T21:04:05.813ZDEBGnexus (BackgroundTasks): activation complete background_task = attached_subnet_manager elapsed = 237.113734ms iteration = 8 status = {"db_error":null,"dendrite":{"switch0":{"errors":[],"n_subnets_added":0,"n_subnets_removed":0,"n_total_subnets":0}},"sled":{}}
80622026-03-24T21:04:05.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80632026-03-24T21:04:05.817ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "eddd88c2-eb37-4e39-b98e-a9b8c6860059", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} })
80642026-03-24T21:04:05.817ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
80652026-03-24T21:04:05.817ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
80662026-03-24T21:04:05.817ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 241.234525ms iteration = 8 status = {"errors":[],"probes_by_sled":{"b4293d4f-3308-4dfa-b261-a103215913e4":0}}
80672026-03-24T21:04:05.817ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Dependency
80682026-03-24T21:04:05.817ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
80692026-03-24T21:04:05.817ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80702026-03-24T21:04:05.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80712026-03-24T21:04:05.819ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80722026-03-24T21:04:05.820ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
80732026-03-24T21:04:05.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80742026-03-24T21:04:05.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80752026-03-24T21:04:05.821ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
80762026-03-24T21:04:05.821ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80772026-03-24T21:04:05.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80782026-03-24T21:04:05.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80792026-03-24T21:04:05.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80802026-03-24T21:04:05.823ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80812026-03-24T21:04:05.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80822026-03-24T21:04:05.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80832026-03-24T21:04:05.826ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
80842026-03-24T21:04:05.826ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80852026-03-24T21:04:05.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80862026-03-24T21:04:05.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80872026-03-24T21:04:05.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80882026-03-24T21:04:05.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80892026-03-24T21:04:05.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80902026-03-24T21:04:05.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80912026-03-24T21:04:05.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80922026-03-24T21:04:05.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80932026-03-24T21:04:05.831ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
80942026-03-24T21:04:05.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80952026-03-24T21:04:05.831ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
80962026-03-24T21:04:05.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80972026-03-24T21:04:05.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80982026-03-24T21:04:05.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80992026-03-24T21:04:05.835ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
81002026-03-24T21:04:05.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
81012026-03-24T21:04:05.836ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
81022026-03-24T21:04:05.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81032026-03-24T21:04:05.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81042026-03-24T21:04:05.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81052026-03-24T21:04:05.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81062026-03-24T21:04:05.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81072026-03-24T21:04:05.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81082026-03-24T21:04:05.838ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "52fa8ee8-8aa1-45e3-b1a7-3c3d1e168be3", "content-length": "372", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} })
81092026-03-24T21:04:05.838ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81102026-03-24T21:04:05.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81112026-03-24T21:04:05.843ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
81122026-03-24T21:04:05.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81132026-03-24T21:04:05.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81142026-03-24T21:04:05.844ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 311.470669ms iteration = 25 status = {"physical_disks_added":0}
81152026-03-24T21:04:05.845ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
81162026-03-24T21:04:05.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81172026-03-24T21:04:05.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81182026-03-24T21:04:05.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81192026-03-24T21:04:05.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81202026-03-24T21:04:05.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81212026-03-24T21:04:05.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81222026-03-24T21:04:05.848ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81232026-03-24T21:04:05.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81242026-03-24T21:04:05.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81252026-03-24T21:04:05.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81262026-03-24T21:04:05.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81272026-03-24T21:04:05.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81282026-03-24T21:04:05.855ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
81292026-03-24T21:04:05.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81302026-03-24T21:04:05.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81312026-03-24T21:04:05.856ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
81322026-03-24T21:04:05.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81332026-03-24T21:04:05.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81342026-03-24T21:04:05.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81352026-03-24T21:04:05.858ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
81362026-03-24T21:04:05.858ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
81372026-03-24T21:04:05.860ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "33ced23c-3253-4abb-88a4-3e47198d3264", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} })
81382026-03-24T21:04:05.860ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81392026-03-24T21:04:05.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81402026-03-24T21:04:05.862ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
81412026-03-24T21:04:05.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81422026-03-24T21:04:05.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81432026-03-24T21:04:05.863ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81442026-03-24T21:04:05.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81452026-03-24T21:04:05.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81462026-03-24T21:04:05.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81472026-03-24T21:04:05.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81482026-03-24T21:04:05.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81492026-03-24T21:04:05.868ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81502026-03-24T21:04:05.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81512026-03-24T21:04:05.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81522026-03-24T21:04:05.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81532026-03-24T21:04:05.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81542026-03-24T21:04:05.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81552026-03-24T21:04:05.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81562026-03-24T21:04:05.873ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
81572026-03-24T21:04:05.873ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
81582026-03-24T21:04:05.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81592026-03-24T21:04:05.873ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "859623c6-a69f-4a80-85b7-82e176789817", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} })
81602026-03-24T21:04:05.873ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 56.623367ms iteration = 23 status = {}
81612026-03-24T21:04:05.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81622026-03-24T21:04:05.874ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "dc7ae53d-64b0-4529-a1bc-60ccfa36a04d", "date": "Tue, 24 Mar 2026 21:04:05 GMT"} })
81632026-03-24T21:04:05.874ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
81642026-03-24T21:04:05.874ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81652026-03-24T21:04:05.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81662026-03-24T21:04:05.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81672026-03-24T21:04:05.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81682026-03-24T21:04:05.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81692026-03-24T21:04:05.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81702026-03-24T21:04:05.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81712026-03-24T21:04:05.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
81722026-03-24T21:04:05.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81732026-03-24T21:04:05.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81742026-03-24T21:04:05.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81752026-03-24T21:04:05.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81762026-03-24T21:04:05.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81772026-03-24T21:04:05.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81782026-03-24T21:04:05.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81792026-03-24T21:04:05.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81802026-03-24T21:04:05.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81812026-03-24T21:04:05.886ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
81822026-03-24T21:04:05.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
81832026-03-24T21:04:05.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81842026-03-24T21:04:05.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81852026-03-24T21:04:05.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81862026-03-24T21:04:05.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81872026-03-24T21:04:05.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81882026-03-24T21:04:05.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81892026-03-24T21:04:05.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81902026-03-24T21:04:05.901ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
81912026-03-24T21:04:05.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81922026-03-24T21:04:05.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81932026-03-24T21:04:05.907ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
81942026-03-24T21:04:05.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81952026-03-24T21:04:05.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
81962026-03-24T21:04:05.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81972026-03-24T21:04:05.911ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
81982026-03-24T21:04:05.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
81992026-03-24T21:04:05.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82002026-03-24T21:04:05.913ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
82012026-03-24T21:04:05.913ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
82022026-03-24T21:04:05.913ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
82032026-03-24T21:04:05.920ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82042026-03-24T21:04:05.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82052026-03-24T21:04:05.923ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82062026-03-24T21:04:05.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82072026-03-24T21:04:05.925ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
82082026-03-24T21:04:05.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
82092026-03-24T21:04:05.927ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 394.780435ms iteration = 15 status = {}
82102026-03-24T21:04:05.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82112026-03-24T21:04:05.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82122026-03-24T21:04:05.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82132026-03-24T21:04:05.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82142026-03-24T21:04:05.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82152026-03-24T21:04:05.954ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
82162026-03-24T21:04:05.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82172026-03-24T21:04:05.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82182026-03-24T21:04:05.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82192026-03-24T21:04:05.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82202026-03-24T21:04:05.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82212026-03-24T21:04:05.961ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
82222026-03-24T21:04:05.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82232026-03-24T21:04:05.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82242026-03-24T21:04:05.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82252026-03-24T21:04:05.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82262026-03-24T21:04:05.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82272026-03-24T21:04:05.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
82282026-03-24T21:04:05.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82292026-03-24T21:04:05.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82302026-03-24T21:04:05.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82312026-03-24T21:04:05.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82322026-03-24T21:04:05.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82332026-03-24T21:04:05.977ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
82342026-03-24T21:04:05.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82352026-03-24T21:04:05.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82362026-03-24T21:04:06.019ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:118 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
82372026-03-24T21:04:06.019ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 460.640481ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
82382026-03-24T21:04:06.036ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
82392026-03-24T21:04:06.036ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
82402026-03-24T21:04:06.036ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 66ab1768-c81a-4850-b600-36fe64cea1a5 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
82412026-03-24T21:04:06.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82422026-03-24T21:04:06.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82432026-03-24T21:04:06.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82442026-03-24T21:04:06.041ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
82452026-03-24T21:04:06.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82462026-03-24T21:04:06.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82472026-03-24T21:04:06.044ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
82482026-03-24T21:04:06.044ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
82492026-03-24T21:04:06.044ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 512.043787ms iteration = 20 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
82502026-03-24T21:04:06.094ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
82512026-03-24T21:04:06.094ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
82522026-03-24T21:04:06.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82532026-03-24T21:04:06.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82542026-03-24T21:04:06.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82552026-03-24T21:04:06.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82562026-03-24T21:04:06.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82572026-03-24T21:04:06.105ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
82582026-03-24T21:04:06.105ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
82592026-03-24T21:04:06.105ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "84d50a60-4fbf-4b34-927b-de1641b2ee44", "date": "Tue, 24 Mar 2026 21:04:06 GMT"} })
82602026-03-24T21:04:06.105ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
82612026-03-24T21:04:06.105ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = Dns step id = Ensure
82622026-03-24T21:04:06.105ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
82632026-03-24T21:04:06.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82642026-03-24T21:04:06.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82652026-03-24T21:04:06.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82662026-03-24T21:04:06.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82672026-03-24T21:04:06.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82682026-03-24T21:04:06.112ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
82692026-03-24T21:04:06.154ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
82702026-03-24T21:04:06.154ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
82712026-03-24T21:04:06.154ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
82722026-03-24T21:04:06.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82732026-03-24T21:04:06.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82742026-03-24T21:04:06.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82752026-03-24T21:04:06.158ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
82762026-03-24T21:04:06.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82772026-03-24T21:04:06.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82782026-03-24T21:04:06.160ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
82792026-03-24T21:04:06.160ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
82802026-03-24T21:04:06.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82812026-03-24T21:04:06.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82822026-03-24T21:04:06.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82832026-03-24T21:04:06.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
82842026-03-24T21:04:06.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82852026-03-24T21:04:06.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82862026-03-24T21:04:06.183ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
82872026-03-24T21:04:06.183ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
82882026-03-24T21:04:06.183ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
82892026-03-24T21:04:06.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82902026-03-24T21:04:06.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82912026-03-24T21:04:06.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82922026-03-24T21:04:06.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82932026-03-24T21:04:06.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82942026-03-24T21:04:06.189ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
82952026-03-24T21:04:06.232ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
82962026-03-24T21:04:06.232ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
82972026-03-24T21:04:06.232ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
82982026-03-24T21:04:06.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82992026-03-24T21:04:06.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83002026-03-24T21:04:06.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83012026-03-24T21:04:06.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
83022026-03-24T21:04:06.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83032026-03-24T21:04:06.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83042026-03-24T21:04:06.279ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
83052026-03-24T21:04:06.279ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
83062026-03-24T21:04:06.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83072026-03-24T21:04:06.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83082026-03-24T21:04:06.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83092026-03-24T21:04:06.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
83102026-03-24T21:04:06.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83112026-03-24T21:04:06.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83122026-03-24T21:04:06.286ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
83132026-03-24T21:04:06.286ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
83142026-03-24T21:04:06.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83152026-03-24T21:04:06.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83162026-03-24T21:04:06.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83172026-03-24T21:04:06.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
83182026-03-24T21:04:06.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83192026-03-24T21:04:06.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83202026-03-24T21:04:06.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83212026-03-24T21:04:06.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83222026-03-24T21:04:06.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83232026-03-24T21:04:06.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83242026-03-24T21:04:06.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83252026-03-24T21:04:06.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
83262026-03-24T21:04:06.300ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
83272026-03-24T21:04:06.300ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
83282026-03-24T21:04:06.300ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = OmicronZones step id = Cleanup
83292026-03-24T21:04:06.300ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = OmicronZones step id = Cleanup
83302026-03-24T21:04:06.300ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = PhysicalDisks step id = Cleanup
83312026-03-24T21:04:06.300ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = Clickhouse step id = Ensure
83322026-03-24T21:04:06.300ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = Clickhouse step id = Ensure
83332026-03-24T21:04:06.300ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
83342026-03-24T21:04:06.305ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "4c542370-ff05-4979-aca9-c8c7840b6965", "date": "Tue, 24 Mar 2026 21:04:06 GMT"} })
83352026-03-24T21:04:06.305ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = SupportBundles step id = Cleanup
83362026-03-24T21:04:06.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83372026-03-24T21:04:06.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83382026-03-24T21:04:06.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83392026-03-24T21:04:06.308ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
83402026-03-24T21:04:06.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83412026-03-24T21:04:06.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83422026-03-24T21:04:06.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83432026-03-24T21:04:06.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83442026-03-24T21:04:06.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83452026-03-24T21:04:06.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
83462026-03-24T21:04:06.317ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = OmicronZones step id = Cleanup
83472026-03-24T21:04:06.317ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
83482026-03-24T21:04:06.317ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
83492026-03-24T21:04:06.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83502026-03-24T21:04:06.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83512026-03-24T21:04:06.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83522026-03-24T21:04:06.320ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
83532026-03-24T21:04:06.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83542026-03-24T21:04:06.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83552026-03-24T21:04:06.370ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
83562026-03-24T21:04:06.370ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9e1c09c1-3960-4ecf-a904-19e137699ab3 (blueprint), false)
83572026-03-24T21:04:06.370ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = Cockroach step id = Ensure
83582026-03-24T21:04:06.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83592026-03-24T21:04:06.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83602026-03-24T21:04:06.598ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
83612026-03-24T21:04:06.598ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 8d20142e-0935-472f-84e0-9aab7d7d327d step component = MgsUpdates step id = Ensure
83622026-03-24T21:04:06.598ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 10 reason = Dependency
83632026-03-24T21:04:06.598ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Dependency
83642026-03-24T21:04:06.598ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83652026-03-24T21:04:06.598ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
83662026-03-24T21:04:06.599ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.077813402s iteration = 9 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3240,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":65673521,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":65673491,"secs":0}},"event_index":1,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":65686751,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":50396479,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":50396439,"secs":0}},"event_index":2,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":116098250,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":107681754,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":107681694,"secs":0}},"event_index":3,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":223800334,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4560573,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4560533,"secs":0}},"event_index":4,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":228380558,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":356301308,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":356301268,"secs":0}},"event_index":5,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":584693936,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":194048883,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":194048853,"secs":0}},"event_index":6,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":778754059,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4670,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4640,"secs":0}},"event_index":7,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":778770399,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2470,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2440,"secs":0}},"event_index":8,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":778774929,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3371,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3340,"secs":0}},"event_index":9,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":778780259,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1520,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1491,"secs":0}},"event_index":10,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":778783190,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5290722,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":5290682,"secs":0}},"event_index":11,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":784075332,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":12237170,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":12237130,"secs":0}},"event_index":12,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":796316312,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":53065571,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":53065541,"secs":0}},"event_index":13,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":849397794,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":228018217,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":228018187,"secs":0}},"event_index":14,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":77425561,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4730,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4700,"secs":0}},"event_index":15,"execution_id":"0434cb74-8bf3-40e3-acc9-86cebd0671fc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":77440442,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3"}
83672026-03-24T21:04:06.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
83682026-03-24T21:04:06.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
83692026-03-24T21:04:06.600ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83702026-03-24T21:04:06.600ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83712026-03-24T21:04:06.603ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
83722026-03-24T21:04:06.603ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
83732026-03-24T21:04:06.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
83742026-03-24T21:04:06.603ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
83752026-03-24T21:04:06.603ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83762026-03-24T21:04:06.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
83772026-03-24T21:04:06.642ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
83782026-03-24T21:04:06.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
83792026-03-24T21:04:06.644ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83802026-03-24T21:04:06.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
83812026-03-24T21:04:06.646ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
83822026-03-24T21:04:06.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
83832026-03-24T21:04:06.648ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
83842026-03-24T21:04:06.648ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 49.31924ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
83852026-03-24T21:04:07.484ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(6d53e1e9-ea3f-4511-bee9-394108b97e31, 0), (9789333b-5bf2-4c58-8e77-63169dbd8708, 0), (32f38600-4055-4a7f-bcbc-7299e531f9f7, 0), (dbfdf733-53f6-4263-bd98-b247cf0b5509, 0)] collection_id = 6d53e1e9-ea3f-4511-bee9-394108b97e31
83862026-03-24T21:04:07.485ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83872026-03-24T21:04:07.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
83882026-03-24T21:04:08.551ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:35434
83892026-03-24T21:04:08.551ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = b52b2bd0-cc20-4542-a893-d1973f540f3c uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
83902026-03-24T21:04:08.551ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = b52b2bd0-cc20-4542-a893-d1973f540f3c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
83912026-03-24T21:04:08.551ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = b52b2bd0-cc20-4542-a893-d1973f540f3c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
83922026-03-24T21:04:08.553ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2633 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = b52b2bd0-cc20-4542-a893-d1973f540f3c response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
83932026-03-24T21:04:08.553ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 918e5032-5582-46ff-af0d-c5f55d19792d uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
83942026-03-24T21:04:08.554ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 918e5032-5582-46ff-af0d-c5f55d19792d roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
83952026-03-24T21:04:08.554ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 918e5032-5582-46ff-af0d-c5f55d19792d resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
83962026-03-24T21:04:08.556ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2623 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 918e5032-5582-46ff-af0d-c5f55d19792d response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
83972026-03-24T21:04:08.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 347 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 4878b424-fb92-4010-bd7d-76755f9d10a7 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
83982026-03-24T21:04:08.576ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 6d53e1e9-ea3f-4511-bee9-394108b97e31 file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
83992026-03-24T21:04:08.576ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
84002026-03-24T21:04:08.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
84012026-03-24T21:04:09.228ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(32f38600-4055-4a7f-bcbc-7299e531f9f7, 0), (dbfdf733-53f6-4263-bd98-b247cf0b5509, 0), (bfcfe115-13a5-4646-a408-f5f6ef741c06, 0)]
84022026-03-24T21:04:09.229ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1235, tv_nsec: 443134141 } })
84032026-03-24T21:04:09.230ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1345db7a-abc1-4a4c-a49c-e6c56102e521.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1920db37-87f5-4a96-b3ef-1db2eb59e9a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6b02d2eb-4373-4f12-8efc-4c0e3f4fa77b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("709c4704-f768-4640-a198-a23621ed6e11.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1235, tv_nsec: 444831257 } })
84042026-03-24T21:04:09.231ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
84052026-03-24T21:04:09.231ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
84062026-03-24T21:04:09.231ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
84072026-03-24T21:04:09.231ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "af2f5cf1-f00a-4116-a71a-5688837c4321", "content-length": "476", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84082026-03-24T21:04:09.231ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
84092026-03-24T21:04:09.232ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6c8e159-ee53-42f5-a1ae-038af538470e", "content-length": "734", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84102026-03-24T21:04:09.232ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
84112026-03-24T21:04:09.232ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2f8b6c2-03d8-4455-b663-680086131ac6", "content-length": "111", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84122026-03-24T21:04:09.232ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
84132026-03-24T21:04:09.233ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfca91ac-e6a3-4f3e-b412-5fc48ce3643b", "content-length": "111", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84142026-03-24T21:04:09.233ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
84152026-03-24T21:04:09.233ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bab59b4b-6c20-4555-807c-b37f3dd64772", "content-length": "163", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84162026-03-24T21:04:09.233ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
84172026-03-24T21:04:09.233ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca77c448-e08a-45f5-9b0b-cbf10318a916", "content-length": "163", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84182026-03-24T21:04:09.233ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
84192026-03-24T21:04:09.234ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a53a3daf-c486-4850-932d-c25bae7d0d00", "content-length": "166", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84202026-03-24T21:04:09.234ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
84212026-03-24T21:04:09.234ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c4504ae-f2e4-42cd-ada3-3a62459bddb8", "content-length": "166", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84222026-03-24T21:04:09.234ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
84232026-03-24T21:04:09.234ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e666703-bb3a-461e-8281-47adaecbfa8c", "content-length": "702", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84242026-03-24T21:04:09.234ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
84252026-03-24T21:04:09.235ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf4eedc6-864b-4986-ba3e-0b7e79af4219", "content-length": "727", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84262026-03-24T21:04:09.235ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
84272026-03-24T21:04:09.235ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2bf305a-7f26-46d6-a550-42d1f7051968", "content-length": "729", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84282026-03-24T21:04:09.235ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
84292026-03-24T21:04:09.235ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "352c5e0f-955e-46fe-94b7-ef2df8a310ae", "content-length": "728", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84302026-03-24T21:04:09.235ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
84312026-03-24T21:04:09.235ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0cdacd4-663a-48d1-a884-8c53c6bd3818", "content-length": "722", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84322026-03-24T21:04:09.235ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
84332026-03-24T21:04:09.236ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "066f71c1-f412-4775-9a04-7bab1ad534b8", "content-length": "10", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84342026-03-24T21:04:09.236ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
84352026-03-24T21:04:09.236ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "34e8d47f-2c7b-4b78-a23f-74a3c26fdbcc", "content-length": "144", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84362026-03-24T21:04:09.236ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
84372026-03-24T21:04:09.236ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "54c04609-be1d-4b81-8d95-43a87a4a651d", "content-length": "147", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84382026-03-24T21:04:09.236ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
84392026-03-24T21:04:09.236ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1c21cc0-ffab-4e3b-bbc4-208879764146", "content-length": "109", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84402026-03-24T21:04:09.236ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
84412026-03-24T21:04:09.237ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdea90bc-0089-4502-8310-e45ac030a91f", "content-length": "109", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84422026-03-24T21:04:09.237ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
84432026-03-24T21:04:09.237ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "333a1a4f-722a-45b3-acc8-5ff61349c5d4", "content-length": "163", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84442026-03-24T21:04:09.237ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
84452026-03-24T21:04:09.238ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f041cb2-a2cb-44bc-a9de-cbf5b79d8a00", "content-length": "163", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84462026-03-24T21:04:09.238ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
84472026-03-24T21:04:09.238ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5416e3c4-df8e-4bf8-9b38-426002e1985d", "content-length": "166", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84482026-03-24T21:04:09.238ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
84492026-03-24T21:04:09.239ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5333caf5-564c-4881-b998-c80c25f63f16", "content-length": "166", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84502026-03-24T21:04:09.239ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
84512026-03-24T21:04:09.239ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "97a1e35e-131d-456a-9bea-0e35da4d3255", "content-length": "702", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84522026-03-24T21:04:09.239ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
84532026-03-24T21:04:09.239ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "648d562f-b77d-4b12-9e23-33019cde44ec", "content-length": "727", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84542026-03-24T21:04:09.239ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
84552026-03-24T21:04:09.239ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c963711-fecf-47e8-9eac-472dbfe4390d", "content-length": "729", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84562026-03-24T21:04:09.239ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
84572026-03-24T21:04:09.239ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77aa5f24-2914-493e-ae79-82126973f8f0", "content-length": "728", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84582026-03-24T21:04:09.239ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
84592026-03-24T21:04:09.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
84602026-03-24T21:04:09.241ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84612026-03-24T21:04:09.243ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
84622026-03-24T21:04:09.243ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
84632026-03-24T21:04:09.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
84642026-03-24T21:04:09.245ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84652026-03-24T21:04:09.248ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
84662026-03-24T21:04:09.248ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
84672026-03-24T21:04:09.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
84682026-03-24T21:04:09.252ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
84692026-03-24T21:04:09.252ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
84702026-03-24T21:04:09.344ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "14809194-4e55-4d19-a479-228cd6e0d6c9", "content-length": "110670", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84712026-03-24T21:04:09.344ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
84722026-03-24T21:04:09.344ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
84732026-03-24T21:04:09.344ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
84742026-03-24T21:04:09.344ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
84752026-03-24T21:04:09.344ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
84762026-03-24T21:04:09.344ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
84772026-03-24T21:04:09.344ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
84782026-03-24T21:04:09.344ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
84792026-03-24T21:04:09.345ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "308a03ae-342e-4489-aa29-79db94a19266", "content-length": "64", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84802026-03-24T21:04:09.345ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c249f60a-ea6c-43d2-99af-885f5446bfa0", "content-length": "64", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84812026-03-24T21:04:09.345ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
84822026-03-24T21:04:09.345ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
84832026-03-24T21:04:09.345ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec8af8ca-dc6d-4bc6-9c22-3caf2a376f32", "content-length": "64", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84842026-03-24T21:04:09.345ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
84852026-03-24T21:04:09.345ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2772fce8-e283-47bf-ba56-0a5671b82c26", "content-length": "64", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84862026-03-24T21:04:09.345ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbb5729a-1d65-4374-83c4-3a396a7581e2", "content-length": "64", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84872026-03-24T21:04:09.345ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
84882026-03-24T21:04:09.345ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
84892026-03-24T21:04:09.353ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d18581e7-95f2-4daf-b37c-1a2a6278480b", "content-length": "427635", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84902026-03-24T21:04:09.353ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "706bb219-5c54-4c2a-a038-ba1afb9b98b7", "content-length": "404972", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84912026-03-24T21:04:09.353ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1850a3af-5357-4f2e-b752-6fef7d58a32a", "content-length": "398102", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84922026-03-24T21:04:09.353ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "686e67cf-387d-413e-9b3e-030dfba0593f", "content-length": "393378", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84932026-03-24T21:04:09.354ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca6445e1-e338-4799-88fd-41df22042513", "content-length": "396981", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
84942026-03-24T21:04:09.357ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
84952026-03-24T21:04:09.357ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
84962026-03-24T21:04:09.357ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
84972026-03-24T21:04:09.357ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
84982026-03-24T21:04:09.357ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
84992026-03-24T21:04:09.358ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 (omicron_zone)
85002026-03-24T21:04:09.359ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
85012026-03-24T21:04:09.362ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa76916f-559a-43f1-8503-fd1358504fbc", "content-length": "128", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
85022026-03-24T21:04:09.362ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
85032026-03-24T21:04:09.364ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
85042026-03-24T21:04:09.364ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
85052026-03-24T21:04:09.365ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "78f5fb73-17a0-49f2-a37a-9ef1c814837c", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
85062026-03-24T21:04:09.365ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
85072026-03-24T21:04:09.365ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
85082026-03-24T21:04:09.365ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
85092026-03-24T21:04:09.365ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ecce173-f15f-4158-a566-04e408529877", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
85102026-03-24T21:04:09.365ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
85112026-03-24T21:04:09.366ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
85122026-03-24T21:04:09.366ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
85132026-03-24T21:04:09.366ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "39a57537-6a8f-4a3c-b2f0-132cef5fc602", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:04:09 GMT"} })
85142026-03-24T21:04:09.366ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
85152026-03-24T21:04:09.366ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
85162026-03-24T21:04:09.366ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
85172026-03-24T21:04:09.366ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
85182026-03-24T21:04:09.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
85192026-03-24T21:04:09.368ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85202026-03-24T21:04:09.370ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
85212026-03-24T21:04:09.370ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
85222026-03-24T21:04:09.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
85232026-03-24T21:04:09.759ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = bfe73e65-b1ea-44ad-b413-8bbb37fec3ef file = nexus/db-queries/src/db/datastore/inventory.rs:1757
85242026-03-24T21:04:09.759ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = bfe73e65-b1ea-44ad-b413-8bbb37fec3ef time_started = 2026-03-24 21:04:09.231336 UTC
85252026-03-24T21:04:09.759ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 39 reason = Dependency
85262026-03-24T21:04:09.759ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
85272026-03-24T21:04:09.759ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 3.160433209s iteration = 10 status = {"collection_id":"bfe73e65-b1ea-44ad-b413-8bbb37fec3ef","time_done":"2026-03-24 21:04:09.366621 UTC","time_started":"2026-03-24 21:04:09.231336 UTC"}
85282026-03-24T21:04:09.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
85292026-03-24T21:04:09.761ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85302026-03-24T21:04:09.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
85312026-03-24T21:04:09.764ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
85322026-03-24T21:04:09.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
85332026-03-24T21:04:10.075ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
85342026-03-24T21:04:10.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
85352026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 19 reason = Dependency
85362026-03-24T21:04:10.120ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
85372026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 21 reason = Dependency
85382026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 361.070636ms iteration = 39 status = {"Loaded":{"collection_id":"bfe73e65-b1ea-44ad-b413-8bbb37fec3ef","time_loaded":"2026-03-24T21:04:09.759223176Z","time_started":"2026-03-24T21:04:09.231336Z"}}
85392026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 17.391\u{b5}s iteration = 19 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
85402026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Dependency
85412026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 13 reason = Dependency
85422026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85432026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85442026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
85452026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85462026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
85472026-03-24T21:04:10.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85482026-03-24T21:04:10.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85492026-03-24T21:04:10.122ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85502026-03-24T21:04:10.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85512026-03-24T21:04:10.125ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
85522026-03-24T21:04:10.125ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
85532026-03-24T21:04:10.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
85542026-03-24T21:04:10.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85552026-03-24T21:04:10.125ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
85562026-03-24T21:04:10.125ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85572026-03-24T21:04:10.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
85582026-03-24T21:04:10.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85592026-03-24T21:04:10.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85602026-03-24T21:04:10.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85612026-03-24T21:04:10.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85622026-03-24T21:04:10.127ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85632026-03-24T21:04:10.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85642026-03-24T21:04:10.128ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85652026-03-24T21:04:10.129ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
85662026-03-24T21:04:10.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85672026-03-24T21:04:10.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85682026-03-24T21:04:10.130ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
85692026-03-24T21:04:10.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
85702026-03-24T21:04:10.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
85712026-03-24T21:04:10.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
85722026-03-24T21:04:10.131ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
85732026-03-24T21:04:10.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
85742026-03-24T21:04:10.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85752026-03-24T21:04:10.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85762026-03-24T21:04:10.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
85772026-03-24T21:04:10.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
85782026-03-24T21:04:10.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85792026-03-24T21:04:10.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85802026-03-24T21:04:10.135ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
85812026-03-24T21:04:10.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85822026-03-24T21:04:10.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85832026-03-24T21:04:10.136ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
85842026-03-24T21:04:10.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
85852026-03-24T21:04:10.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
85862026-03-24T21:04:10.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85872026-03-24T21:04:10.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85882026-03-24T21:04:10.139ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
85892026-03-24T21:04:10.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85902026-03-24T21:04:10.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85912026-03-24T21:04:10.140ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
85922026-03-24T21:04:10.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
85932026-03-24T21:04:10.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
85942026-03-24T21:04:10.146ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
85952026-03-24T21:04:10.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
85962026-03-24T21:04:10.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
85972026-03-24T21:04:10.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85982026-03-24T21:04:10.150ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
85992026-03-24T21:04:10.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
86002026-03-24T21:04:10.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
86012026-03-24T21:04:10.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86022026-03-24T21:04:10.154ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
86032026-03-24T21:04:10.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
86042026-03-24T21:04:10.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
86052026-03-24T21:04:10.157ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
86062026-03-24T21:04:10.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
86072026-03-24T21:04:10.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
86082026-03-24T21:04:10.191ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 70.808989ms iteration = 26 status = {"physical_disks_added":0}
86092026-03-24T21:04:10.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86102026-03-24T21:04:10.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86112026-03-24T21:04:10.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86122026-03-24T21:04:10.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86132026-03-24T21:04:10.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86142026-03-24T21:04:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86152026-03-24T21:04:10.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86162026-03-24T21:04:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86172026-03-24T21:04:10.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86182026-03-24T21:04:10.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86192026-03-24T21:04:10.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86202026-03-24T21:04:10.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
86212026-03-24T21:04:10.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86222026-03-24T21:04:10.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86232026-03-24T21:04:10.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86242026-03-24T21:04:10.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86252026-03-24T21:04:10.214ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
86262026-03-24T21:04:10.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
86272026-03-24T21:04:10.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
86282026-03-24T21:04:10.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86292026-03-24T21:04:10.216ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86302026-03-24T21:04:10.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86312026-03-24T21:04:10.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86322026-03-24T21:04:10.218ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
86332026-03-24T21:04:10.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
86342026-03-24T21:04:10.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
86352026-03-24T21:04:10.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86362026-03-24T21:04:10.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86372026-03-24T21:04:10.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86382026-03-24T21:04:10.220ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
86392026-03-24T21:04:10.220ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 100.669835ms iteration = 13 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"bfe73e65-b1ea-44ad-b413-8bbb37fec3ef","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
86402026-03-24T21:04:10.223ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
86412026-03-24T21:04:10.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86422026-03-24T21:04:10.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86432026-03-24T21:04:10.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86442026-03-24T21:04:10.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86452026-03-24T21:04:10.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86462026-03-24T21:04:10.230ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
86472026-03-24T21:04:10.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86482026-03-24T21:04:10.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86492026-03-24T21:04:10.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86502026-03-24T21:04:10.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86512026-03-24T21:04:10.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86522026-03-24T21:04:10.236ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
86532026-03-24T21:04:10.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86542026-03-24T21:04:10.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86552026-03-24T21:04:10.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86562026-03-24T21:04:10.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86572026-03-24T21:04:10.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86582026-03-24T21:04:10.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86592026-03-24T21:04:10.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86602026-03-24T21:04:10.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86612026-03-24T21:04:10.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86622026-03-24T21:04:10.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86632026-03-24T21:04:10.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86642026-03-24T21:04:10.261ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86652026-03-24T21:04:10.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86662026-03-24T21:04:10.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86672026-03-24T21:04:10.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86682026-03-24T21:04:10.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86692026-03-24T21:04:10.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86702026-03-24T21:04:10.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86712026-03-24T21:04:10.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86722026-03-24T21:04:10.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86732026-03-24T21:04:10.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86742026-03-24T21:04:10.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86752026-03-24T21:04:10.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86762026-03-24T21:04:10.274ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86772026-03-24T21:04:10.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86782026-03-24T21:04:10.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86792026-03-24T21:04:10.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86802026-03-24T21:04:10.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86812026-03-24T21:04:10.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86822026-03-24T21:04:10.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86832026-03-24T21:04:10.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86842026-03-24T21:04:10.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
86852026-03-24T21:04:10.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86862026-03-24T21:04:10.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86872026-03-24T21:04:10.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86882026-03-24T21:04:10.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86892026-03-24T21:04:10.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86902026-03-24T21:04:10.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
86912026-03-24T21:04:10.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86922026-03-24T21:04:10.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86932026-03-24T21:04:10.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86942026-03-24T21:04:10.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86952026-03-24T21:04:10.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86962026-03-24T21:04:10.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
86972026-03-24T21:04:10.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86982026-03-24T21:04:10.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86992026-03-24T21:04:10.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87002026-03-24T21:04:10.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87012026-03-24T21:04:10.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87022026-03-24T21:04:10.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
87032026-03-24T21:04:10.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87042026-03-24T21:04:10.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87052026-03-24T21:04:10.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87062026-03-24T21:04:10.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87072026-03-24T21:04:10.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87082026-03-24T21:04:10.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
87092026-03-24T21:04:10.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87102026-03-24T21:04:10.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87112026-03-24T21:04:10.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87122026-03-24T21:04:10.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87132026-03-24T21:04:10.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87142026-03-24T21:04:10.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
87152026-03-24T21:04:10.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87162026-03-24T21:04:10.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87172026-03-24T21:04:10.313ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
87182026-03-24T21:04:10.313ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
87192026-03-24T21:04:10.313ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 50393f13-57b6-4798-a9de-39894c0c8301 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
87202026-03-24T21:04:10.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87212026-03-24T21:04:10.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87222026-03-24T21:04:10.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87232026-03-24T21:04:10.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
87242026-03-24T21:04:10.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87252026-03-24T21:04:10.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87262026-03-24T21:04:10.319ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
87272026-03-24T21:04:10.319ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
87282026-03-24T21:04:10.319ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 199.393527ms iteration = 21 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
87292026-03-24T21:04:10.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 86 reason = Timeout
87302026-03-24T21:04:10.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87312026-03-24T21:04:10.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87322026-03-24T21:04:10.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87332026-03-24T21:04:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
87342026-03-24T21:04:10.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87352026-03-24T21:04:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87362026-03-24T21:04:10.538ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
87372026-03-24T21:04:10.538ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.712361ms iteration = 86 status = {"config_updated":false}
87382026-03-24T21:04:15.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
87392026-03-24T21:04:15.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 87 reason = Timeout
87402026-03-24T21:04:15.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87412026-03-24T21:04:15.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87422026-03-24T21:04:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87432026-03-24T21:04:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
87442026-03-24T21:04:15.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87452026-03-24T21:04:15.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87462026-03-24T21:04:15.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
87472026-03-24T21:04:15.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87482026-03-24T21:04:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
87492026-03-24T21:04:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
87502026-03-24T21:04:15.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87512026-03-24T21:04:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87522026-03-24T21:04:15.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87532026-03-24T21:04:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
87542026-03-24T21:04:15.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
87552026-03-24T21:04:15.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.882062ms iteration = 87 status = {"config_updated":false}
87562026-03-24T21:04:15.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87572026-03-24T21:04:15.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
87582026-03-24T21:04:15.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
87592026-03-24T21:04:15.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
87602026-03-24T21:04:15.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
87612026-03-24T21:04:15.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
87622026-03-24T21:04:15.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
87632026-03-24T21:04:15.561ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
87642026-03-24T21:04:15.561ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.011103ms iteration = 44 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
87652026-03-24T21:04:15.605ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
87662026-03-24T21:04:15.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
87672026-03-24T21:04:15.607ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 47.084739ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
87682026-03-24T21:04:18.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 416 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = befabbde-8541-4847-9a28-e09913c655bb response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
87692026-03-24T21:04:20.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 40 reason = Timeout
87702026-03-24T21:04:20.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 88 reason = Timeout
87712026-03-24T21:04:20.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
87722026-03-24T21:04:20.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87732026-03-24T21:04:20.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87742026-03-24T21:04:20.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
87752026-03-24T21:04:20.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87762026-03-24T21:04:20.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87772026-03-24T21:04:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
87782026-03-24T21:04:20.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
87792026-03-24T21:04:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
87802026-03-24T21:04:20.539ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
87812026-03-24T21:04:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
87822026-03-24T21:04:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
87832026-03-24T21:04:20.541ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = bfe73e65-b1ea-44ad-b413-8bbb37fec3ef original_time_started = 2026-03-24 21:04:09.231336 UTC
87842026-03-24T21:04:20.541ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.102659ms iteration = 40 status = {"Loaded":{"collection_id":"bfe73e65-b1ea-44ad-b413-8bbb37fec3ef","time_loaded":"2026-03-24T21:04:20.533248610Z","time_started":"2026-03-24T21:04:09.231336Z"}}
87852026-03-24T21:04:20.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
87862026-03-24T21:04:20.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.601662ms iteration = 88 status = {"config_updated":false}
87872026-03-24T21:04:20.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 30 reason = Timeout
87882026-03-24T21:04:20.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
87892026-03-24T21:04:20.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 30 reason = Timeout
87902026-03-24T21:04:20.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
87912026-03-24T21:04:20.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
87922026-03-24T21:04:20.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
87932026-03-24T21:04:20.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
87942026-03-24T21:04:20.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
87952026-03-24T21:04:20.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87962026-03-24T21:04:20.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87972026-03-24T21:04:20.580ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
87982026-03-24T21:04:20.580ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
87992026-03-24T21:04:20.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
88002026-03-24T21:04:20.580ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
88012026-03-24T21:04:20.582ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.029222ms iteration = 30 status = "NoSitrep"
88022026-03-24T21:04:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
88032026-03-24T21:04:20.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
88042026-03-24T21:04:20.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
88052026-03-24T21:04:20.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
88062026-03-24T21:04:20.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88072026-03-24T21:04:20.585ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
88082026-03-24T21:04:20.589ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
88092026-03-24T21:04:20.589ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
88102026-03-24T21:04:20.589ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.258198ms iteration = 30 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
88112026-03-24T21:04:23.552ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 40fd1f48-2777-44c6-b57c-d8ef5b99260e uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
88122026-03-24T21:04:23.552ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 40fd1f48-2777-44c6-b57c-d8ef5b99260e roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
88132026-03-24T21:04:23.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 40fd1f48-2777-44c6-b57c-d8ef5b99260e resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
88142026-03-24T21:04:23.556ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 3657 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 40fd1f48-2777-44c6-b57c-d8ef5b99260e response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
88152026-03-24T21:04:23.556ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = fadd0f6e-3f02-448d-8cbf-a24f1f998dba uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
88162026-03-24T21:04:23.556ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = fadd0f6e-3f02-448d-8cbf-a24f1f998dba roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
88172026-03-24T21:04:23.556ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = fadd0f6e-3f02-448d-8cbf-a24f1f998dba resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
88182026-03-24T21:04:23.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2951 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = fadd0f6e-3f02-448d-8cbf-a24f1f998dba response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
88192026-03-24T21:04:25.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
88202026-03-24T21:04:25.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 89 reason = Timeout
88212026-03-24T21:04:25.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88222026-03-24T21:04:25.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
88232026-03-24T21:04:25.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
88242026-03-24T21:04:25.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
88252026-03-24T21:04:25.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88262026-03-24T21:04:25.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88272026-03-24T21:04:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
88282026-03-24T21:04:25.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88292026-03-24T21:04:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
88302026-03-24T21:04:25.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
88312026-03-24T21:04:25.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
88322026-03-24T21:04:25.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
88332026-03-24T21:04:25.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88342026-03-24T21:04:25.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
88352026-03-24T21:04:25.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
88362026-03-24T21:04:25.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.518522ms iteration = 89 status = {"config_updated":false}
88372026-03-24T21:04:25.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88382026-03-24T21:04:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
88392026-03-24T21:04:25.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88402026-03-24T21:04:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
88412026-03-24T21:04:25.558ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
88422026-03-24T21:04:25.558ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.116037ms iteration = 45 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
88432026-03-24T21:04:25.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
88442026-03-24T21:04:25.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88452026-03-24T21:04:25.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88462026-03-24T21:04:25.563ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
88472026-03-24T21:04:25.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
88482026-03-24T21:04:25.564ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.254318ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
88492026-03-24T21:04:28.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 407 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 6230120a-3174-43fa-89c6-c17cd6139b18 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
88502026-03-24T21:04:30.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 90 reason = Timeout
88512026-03-24T21:04:30.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
88522026-03-24T21:04:30.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
88532026-03-24T21:04:30.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88542026-03-24T21:04:30.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
88552026-03-24T21:04:30.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
88562026-03-24T21:04:30.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
88572026-03-24T21:04:30.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
88582026-03-24T21:04:30.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.349491ms iteration = 90 status = {"config_updated":false}
88592026-03-24T21:04:35.521ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 16 reason = Timeout
88602026-03-24T21:04:35.521ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
88612026-03-24T21:04:35.521ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
88622026-03-24T21:04:35.521ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
88632026-03-24T21:04:35.521ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
88642026-03-24T21:04:35.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
88652026-03-24T21:04:35.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
88662026-03-24T21:04:35.525ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1261, tv_nsec: 739290080 } })
88672026-03-24T21:04:35.527ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
88682026-03-24T21:04:35.527ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 5.840738ms iteration = 16 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
88692026-03-24T21:04:35.527ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1261, tv_nsec: 741405247 } })
88702026-03-24T21:04:35.528ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
88712026-03-24T21:04:35.528ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1261, tv_nsec: 742097686 } })
88722026-03-24T21:04:35.530ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
88732026-03-24T21:04:35.530ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
88742026-03-24T21:04:35.531ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "de35f134-09fd-46ab-ba87-f13f59aee37c", "content-length": "26", "date": "Tue, 24 Mar 2026 21:04:35 GMT"} })
88752026-03-24T21:04:35.531ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
88762026-03-24T21:04:35.531ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
88772026-03-24T21:04:35.531ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1261, tv_nsec: 744865990 } })
88782026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
88792026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
88802026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
88812026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88822026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
88832026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
88842026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
88852026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
88862026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
88872026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
88882026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
88892026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
88902026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 41 reason = Timeout
88912026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
88922026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 91 reason = Timeout
88932026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
88942026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
88952026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88962026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
88972026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
88982026-03-24T21:04:35.533ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 12.198239ms iteration = 16 status = {}
88992026-03-24T21:04:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
89002026-03-24T21:04:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89012026-03-24T21:04:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
89022026-03-24T21:04:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
89032026-03-24T21:04:35.534ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 12.913378ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
89042026-03-24T21:04:35.535ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89052026-03-24T21:04:35.536ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89062026-03-24T21:04:35.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89072026-03-24T21:04:35.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89082026-03-24T21:04:35.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89092026-03-24T21:04:35.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
89102026-03-24T21:04:35.537ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89112026-03-24T21:04:35.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89122026-03-24T21:04:35.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89132026-03-24T21:04:35.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
89142026-03-24T21:04:35.539ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
89152026-03-24T21:04:35.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
89162026-03-24T21:04:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
89172026-03-24T21:04:35.540ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89182026-03-24T21:04:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89192026-03-24T21:04:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
89202026-03-24T21:04:35.541ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89212026-03-24T21:04:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
89222026-03-24T21:04:35.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89232026-03-24T21:04:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89242026-03-24T21:04:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
89252026-03-24T21:04:35.542ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
89262026-03-24T21:04:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
89272026-03-24T21:04:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
89282026-03-24T21:04:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
89292026-03-24T21:04:35.542ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
89302026-03-24T21:04:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
89312026-03-24T21:04:35.543ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89322026-03-24T21:04:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89332026-03-24T21:04:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
89342026-03-24T21:04:35.544ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
89352026-03-24T21:04:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
89362026-03-24T21:04:35.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89372026-03-24T21:04:35.545ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = bfe73e65-b1ea-44ad-b413-8bbb37fec3ef original_time_started = 2026-03-24 21:04:09.231336 UTC
89382026-03-24T21:04:35.545ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 12.038024ms iteration = 41 status = {"Loaded":{"collection_id":"bfe73e65-b1ea-44ad-b413-8bbb37fec3ef","time_loaded":"2026-03-24T21:04:35.533552269Z","time_started":"2026-03-24T21:04:09.231336Z"}}
89392026-03-24T21:04:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
89402026-03-24T21:04:35.546ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
89412026-03-24T21:04:35.546ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 12.171188ms iteration = 91 status = {"config_updated":false}
89422026-03-24T21:04:35.547ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
89432026-03-24T21:04:35.547ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
89442026-03-24T21:04:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
89452026-03-24T21:04:35.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89462026-03-24T21:04:35.548ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89472026-03-24T21:04:35.548ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 15.746695ms iteration = 16 status = 0
89482026-03-24T21:04:35.550ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1261, tv_nsec: 763618476 } })
89492026-03-24T21:04:35.550ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
89502026-03-24T21:04:35.550ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1261, tv_nsec: 764369676 } })
89512026-03-24T21:04:35.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
89522026-03-24T21:04:35.551ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
89532026-03-24T21:04:35.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
89542026-03-24T21:04:35.552ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
89552026-03-24T21:04:35.552ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
89562026-03-24T21:04:35.552ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
89572026-03-24T21:04:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
89582026-03-24T21:04:35.552ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f84316d-f69f-4ca4-8443-e11cb2fef119", "content-length": "26", "date": "Tue, 24 Mar 2026 21:04:35 GMT"} })
89592026-03-24T21:04:35.552ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
89602026-03-24T21:04:35.552ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
89612026-03-24T21:04:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
89622026-03-24T21:04:35.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
89632026-03-24T21:04:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
89642026-03-24T21:04:35.554ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1261, tv_nsec: 767493621 } })
89652026-03-24T21:04:35.555ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1261, tv_nsec: 769372211 } })
89662026-03-24T21:04:35.557ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89672026-03-24T21:04:35.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89682026-03-24T21:04:35.558ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 16 reason = Timeout
89692026-03-24T21:04:35.558ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
89702026-03-24T21:04:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
89712026-03-24T21:04:35.559ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
89722026-03-24T21:04:35.559ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89732026-03-24T21:04:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89742026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
89752026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
89762026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
89772026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
89782026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
89792026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
89802026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
89812026-03-24T21:04:35.560ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
89822026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
89832026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
89842026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
89852026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89862026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
89872026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
89882026-03-24T21:04:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
89892026-03-24T21:04:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
89902026-03-24T21:04:35.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
89912026-03-24T21:04:35.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
89922026-03-24T21:04:35.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
89932026-03-24T21:04:35.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
89942026-03-24T21:04:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
89952026-03-24T21:04:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
89962026-03-24T21:04:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89972026-03-24T21:04:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
89982026-03-24T21:04:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
89992026-03-24T21:04:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
90002026-03-24T21:04:35.562ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90012026-03-24T21:04:35.564ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
90022026-03-24T21:04:35.564ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
90032026-03-24T21:04:35.564ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
90042026-03-24T21:04:35.564ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 6.681441ms iteration = 16 status = {"errors":[],"requests_created_ok":[]}
90052026-03-24T21:04:35.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
90062026-03-24T21:04:35.565ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 31.846418ms iteration = 27 status = {"physical_disks_added":0}
90072026-03-24T21:04:35.565ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
90082026-03-24T21:04:35.565ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
90092026-03-24T21:04:35.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
90102026-03-24T21:04:35.567ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
90112026-03-24T21:04:35.567ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
90122026-03-24T21:04:35.567ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
90132026-03-24T21:04:35.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
90142026-03-24T21:04:35.571ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
90152026-03-24T21:04:35.571ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 37.932103ms iteration = 46 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
90162026-03-24T21:04:35.575ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 16 reason = Timeout
90172026-03-24T21:04:35.576ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 17.29\u{b5}s iteration = 16 status = {"disabled":true,"errors":[],"sps":[]}
90182026-03-24T21:04:35.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 31 reason = Timeout
90192026-03-24T21:04:35.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
90202026-03-24T21:04:35.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 31 reason = Timeout
90212026-03-24T21:04:35.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
90222026-03-24T21:04:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
90232026-03-24T21:04:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
90242026-03-24T21:04:35.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
90252026-03-24T21:04:35.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
90262026-03-24T21:04:35.581ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90272026-03-24T21:04:35.581ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90282026-03-24T21:04:35.583ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
90292026-03-24T21:04:35.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
90302026-03-24T21:04:35.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
90312026-03-24T21:04:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
90322026-03-24T21:04:35.591ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
90332026-03-24T21:04:35.592ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
90342026-03-24T21:04:35.592ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
90352026-03-24T21:04:35.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
90362026-03-24T21:04:35.592ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 16.374051ms iteration = 31 status = "NoSitrep"
90372026-03-24T21:04:35.593ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
90382026-03-24T21:04:35.593ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
90392026-03-24T21:04:35.593ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
90402026-03-24T21:04:35.593ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
90412026-03-24T21:04:35.593ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
90422026-03-24T21:04:35.593ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90432026-03-24T21:04:35.593ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
90442026-03-24T21:04:35.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90452026-03-24T21:04:35.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
90462026-03-24T21:04:35.596ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
90472026-03-24T21:04:35.596ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90482026-03-24T21:04:35.596ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
90492026-03-24T21:04:35.597ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
90502026-03-24T21:04:35.602ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
90512026-03-24T21:04:35.602ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
90522026-03-24T21:04:35.602ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 26.854464ms iteration = 31 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
90532026-03-24T21:04:35.609ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90542026-03-24T21:04:35.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90552026-03-24T21:04:35.612ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90562026-03-24T21:04:35.615ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
90572026-03-24T21:04:35.615ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90582026-03-24T21:04:35.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90592026-03-24T21:04:35.617ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
90602026-03-24T21:04:35.617ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 84.125528ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
90612026-03-24T21:04:35.620ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
90622026-03-24T21:04:35.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
90632026-03-24T21:04:35.621ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 61.472138ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
90642026-03-24T21:04:35.622ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90652026-03-24T21:04:35.623ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
90662026-03-24T21:04:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
90672026-03-24T21:04:35.626ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 79.298148ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
90682026-03-24T21:04:35.627ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
90692026-03-24T21:04:35.627ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
90702026-03-24T21:04:35.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
90712026-03-24T21:04:35.627ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
90722026-03-24T21:04:35.627ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90732026-03-24T21:04:35.627ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
90742026-03-24T21:04:35.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
90752026-03-24T21:04:35.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
90762026-03-24T21:04:35.629ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90772026-03-24T21:04:35.629ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 68.764824ms iteration = 16 status = {"errors":[],"finish_invoked_ok":[]}
90782026-03-24T21:04:35.631ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
90792026-03-24T21:04:35.631ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90802026-03-24T21:04:35.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90812026-03-24T21:04:35.631ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
90822026-03-24T21:04:35.631ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
90832026-03-24T21:04:35.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
90842026-03-24T21:04:35.632ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
90852026-03-24T21:04:35.632ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
90862026-03-24T21:04:35.632ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 71.875488ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
90872026-03-24T21:04:35.634ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
90882026-03-24T21:04:35.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
90892026-03-24T21:04:35.637ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
90902026-03-24T21:04:35.637ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 76.577395ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
90912026-03-24T21:04:35.676ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 116.097481ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
90922026-03-24T21:04:35.684ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 123.251143ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
90932026-03-24T21:04:35.684ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
90942026-03-24T21:04:35.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
90952026-03-24T21:04:35.686ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90962026-03-24T21:04:35.688ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
90972026-03-24T21:04:35.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
90982026-03-24T21:04:35.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
90992026-03-24T21:04:35.693ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
91002026-03-24T21:04:35.693ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
91012026-03-24T21:04:35.694ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5047cebf-074e-42de-8362-429f59d91311", "content-length": "372", "date": "Tue, 24 Mar 2026 21:04:35 GMT"} })
91022026-03-24T21:04:35.694ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91032026-03-24T21:04:35.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91042026-03-24T21:04:35.708ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
91052026-03-24T21:04:35.708ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
91062026-03-24T21:04:35.709ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "15d24a54-08e0-492b-b6af-61d411b20f37", "date": "Tue, 24 Mar 2026 21:04:35 GMT"} })
91072026-03-24T21:04:35.709ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91082026-03-24T21:04:35.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91092026-03-24T21:04:35.712ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91102026-03-24T21:04:35.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91112026-03-24T21:04:35.716ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
91122026-03-24T21:04:35.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
91132026-03-24T21:04:35.719ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
91142026-03-24T21:04:35.719ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
91152026-03-24T21:04:35.726ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ad4dc229-17cb-49f0-9241-612eaf171e6d", "date": "Tue, 24 Mar 2026 21:04:35 GMT"} })
91162026-03-24T21:04:35.726ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 165.483722ms iteration = 24 status = {}
91172026-03-24T21:04:35.726ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
91182026-03-24T21:04:35.727ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2d63827-e79b-4a5e-bf63-1a2f8dccc788", "content-length": "691", "date": "Tue, 24 Mar 2026 21:04:35 GMT"} })
91192026-03-24T21:04:35.727ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
91202026-03-24T21:04:35.727ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
91212026-03-24T21:04:35.727ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
91222026-03-24T21:04:35.727ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 193.934869ms iteration = 16 status = {}
91232026-03-24T21:04:35.735ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 174.661059ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
91242026-03-24T21:04:35.856ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "342964ce-3446-4780-8e45-d5f6e4418534", "date": "Tue, 24 Mar 2026 21:04:35 GMT"} })
91252026-03-24T21:04:35.856ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
91262026-03-24T21:04:35.856ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91272026-03-24T21:04:35.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
91282026-03-24T21:04:35.906ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91292026-03-24T21:04:35.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
91302026-03-24T21:04:35.909ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91312026-03-24T21:04:35.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
91322026-03-24T21:04:35.911ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
91332026-03-24T21:04:35.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
91342026-03-24T21:04:35.913ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 380.661825ms iteration = 16 status = {}
91352026-03-24T21:04:38.551ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 15a046fb-d6a9-4565-a8b3-11d9578cf724 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
91362026-03-24T21:04:38.551ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 15a046fb-d6a9-4565-a8b3-11d9578cf724 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
91372026-03-24T21:04:38.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 15a046fb-d6a9-4565-a8b3-11d9578cf724 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
91382026-03-24T21:04:38.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 3367 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 15a046fb-d6a9-4565-a8b3-11d9578cf724 response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
91392026-03-24T21:04:38.555ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 5986b91d-7137-4b94-8a91-ed9f94410549 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
91402026-03-24T21:04:38.555ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 5986b91d-7137-4b94-8a91-ed9f94410549 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
91412026-03-24T21:04:38.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 5986b91d-7137-4b94-8a91-ed9f94410549 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
91422026-03-24T21:04:38.558ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2789 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 5986b91d-7137-4b94-8a91-ed9f94410549 response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
91432026-03-24T21:04:38.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 354 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 43942f24-0b65-42a0-8111-d53c29b3fdca response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
91442026-03-24T21:04:40.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 92 reason = Timeout
91452026-03-24T21:04:40.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
91462026-03-24T21:04:40.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
91472026-03-24T21:04:40.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91482026-03-24T21:04:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
91492026-03-24T21:04:40.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
91502026-03-24T21:04:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
91512026-03-24T21:04:40.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
91522026-03-24T21:04:40.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.883775ms iteration = 92 status = {"config_updated":false}
91532026-03-24T21:04:45.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
91542026-03-24T21:04:45.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 93 reason = Timeout
91552026-03-24T21:04:45.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91562026-03-24T21:04:45.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
91572026-03-24T21:04:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
91582026-03-24T21:04:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
91592026-03-24T21:04:45.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91602026-03-24T21:04:45.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91612026-03-24T21:04:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
91622026-03-24T21:04:45.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
91632026-03-24T21:04:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
91642026-03-24T21:04:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
91652026-03-24T21:04:45.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91662026-03-24T21:04:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
91672026-03-24T21:04:45.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
91682026-03-24T21:04:45.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.455731ms iteration = 93 status = {"config_updated":false}
91692026-03-24T21:04:45.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91702026-03-24T21:04:45.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
91712026-03-24T21:04:45.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91722026-03-24T21:04:45.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
91732026-03-24T21:04:45.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
91742026-03-24T21:04:45.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
91752026-03-24T21:04:45.558ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
91762026-03-24T21:04:45.558ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.417535ms iteration = 47 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
91772026-03-24T21:04:45.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
91782026-03-24T21:04:45.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91792026-03-24T21:04:45.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
91802026-03-24T21:04:45.563ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
91812026-03-24T21:04:45.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
91822026-03-24T21:04:45.565ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.433873ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
91832026-03-24T21:04:48.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 635 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 77466842-b45f-438b-8be8-bc34e7ceee8c response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
91842026-03-24T21:04:50.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 42 reason = Timeout
91852026-03-24T21:04:50.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 94 reason = Timeout
91862026-03-24T21:04:50.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
91872026-03-24T21:04:50.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
91882026-03-24T21:04:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
91892026-03-24T21:04:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
91902026-03-24T21:04:50.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91912026-03-24T21:04:50.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91922026-03-24T21:04:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
91932026-03-24T21:04:50.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
91942026-03-24T21:04:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
91952026-03-24T21:04:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
91962026-03-24T21:04:50.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
91972026-03-24T21:04:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
91982026-03-24T21:04:50.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
91992026-03-24T21:04:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.160973ms iteration = 94 status = {"config_updated":false}
92002026-03-24T21:04:50.540ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = bfe73e65-b1ea-44ad-b413-8bbb37fec3ef original_time_started = 2026-03-24 21:04:09.231336 UTC
92012026-03-24T21:04:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.685957ms iteration = 42 status = {"Loaded":{"collection_id":"bfe73e65-b1ea-44ad-b413-8bbb37fec3ef","time_loaded":"2026-03-24T21:04:50.532978421Z","time_started":"2026-03-24T21:04:09.231336Z"}}
92022026-03-24T21:04:50.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 32 reason = Timeout
92032026-03-24T21:04:50.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
92042026-03-24T21:04:50.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 32 reason = Timeout
92052026-03-24T21:04:50.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
92062026-03-24T21:04:50.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
92072026-03-24T21:04:50.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
92082026-03-24T21:04:50.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
92092026-03-24T21:04:50.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
92102026-03-24T21:04:50.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92112026-03-24T21:04:50.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92122026-03-24T21:04:50.580ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
92132026-03-24T21:04:50.580ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
92142026-03-24T21:04:50.580ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
92152026-03-24T21:04:50.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
92162026-03-24T21:04:50.582ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.868439ms iteration = 32 status = "NoSitrep"
92172026-03-24T21:04:50.586ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
92182026-03-24T21:04:50.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
92192026-03-24T21:04:50.587ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
92202026-03-24T21:04:50.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
92212026-03-24T21:04:50.589ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92222026-03-24T21:04:50.590ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
92232026-03-24T21:04:50.593ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
92242026-03-24T21:04:50.593ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
92252026-03-24T21:04:50.593ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 17.207674ms iteration = 32 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
92262026-03-24T21:04:55.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
92272026-03-24T21:04:55.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 95 reason = Timeout
92282026-03-24T21:04:55.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92292026-03-24T21:04:55.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
92302026-03-24T21:04:55.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
92312026-03-24T21:04:55.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
92322026-03-24T21:04:55.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92332026-03-24T21:04:55.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92342026-03-24T21:04:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
92352026-03-24T21:04:55.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
92362026-03-24T21:04:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
92372026-03-24T21:04:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
92382026-03-24T21:04:55.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92392026-03-24T21:04:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
92402026-03-24T21:04:55.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
92412026-03-24T21:04:55.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.033ms iteration = 95 status = {"config_updated":false}
92422026-03-24T21:04:55.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92432026-03-24T21:04:55.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
92442026-03-24T21:04:55.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92452026-03-24T21:04:55.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
92462026-03-24T21:04:55.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
92472026-03-24T21:04:55.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
92482026-03-24T21:04:55.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
92492026-03-24T21:04:55.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
92502026-03-24T21:04:55.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
92512026-03-24T21:04:55.562ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
92522026-03-24T21:04:55.562ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.502765ms iteration = 48 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
92532026-03-24T21:04:55.563ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
92542026-03-24T21:04:55.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
92552026-03-24T21:04:55.601ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.633789ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
92562026-03-24T21:04:58.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 359 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 4cbdae1a-ba3e-470f-9799-4007bef1173d response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
92572026-03-24T21:05:00.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 96 reason = Timeout
92582026-03-24T21:05:00.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
92592026-03-24T21:05:00.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
92602026-03-24T21:05:00.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92612026-03-24T21:05:00.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
92622026-03-24T21:05:00.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
92632026-03-24T21:05:00.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
92642026-03-24T21:05:00.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
92652026-03-24T21:05:00.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.471914ms iteration = 96 status = {"config_updated":false}
92662026-03-24T21:05:05.516ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
92672026-03-24T21:05:05.516ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
92682026-03-24T21:05:05.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
92692026-03-24T21:05:05.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
92702026-03-24T21:05:05.517ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
92712026-03-24T21:05:05.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
92722026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
92732026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
92742026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
92752026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 17 reason = Timeout
92762026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
92772026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
92782026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
92792026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
92802026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
92812026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
92822026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
92832026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
92842026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
92852026-03-24T21:05:05.521ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2026-03-24 20:55:05.521072670 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
92862026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
92872026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
92882026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
92892026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
92902026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
92912026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
92922026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
92932026-03-24T21:05:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("93b6e989-1f7d-4884-aec9-ecbc076ed6c4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 735113747 } })
92942026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
92952026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
92962026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
92972026-03-24T21:05:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("27f3a9c5-67c7-41c5-994d-9db513cda557.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2f7ea2ed-fd45-4cd7-9fb1-f48119541118.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 735236601 } })
92982026-03-24T21:05:05.521ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
92992026-03-24T21:05:05.521ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = ExternalNetworking step id = Ensure
93002026-03-24T21:05:05.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
93012026-03-24T21:05:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
93022026-03-24T21:05:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
93032026-03-24T21:05:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.418848ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
93042026-03-24T21:05:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
93052026-03-24T21:05:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.384198ms iteration = 9 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
93062026-03-24T21:05:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
93072026-03-24T21:05:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
93082026-03-24T21:05:05.523ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
93092026-03-24T21:05:05.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
93102026-03-24T21:05:05.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
93112026-03-24T21:05:05.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
93122026-03-24T21:05:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "c018b393-a92f-427e-a3d1-79ae91f4d22a", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
93132026-03-24T21:05:05.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
93142026-03-24T21:05:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b18eb2a9-5dd6-4247-b850-ebf016b1e2e5", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
93152026-03-24T21:05:05.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
93162026-03-24T21:05:05.524ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
93172026-03-24T21:05:05.524ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 3.384401ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
93182026-03-24T21:05:05.525ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 738541630 } })
93192026-03-24T21:05:05.525ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
93202026-03-24T21:05:05.526ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 739631109 } })
93212026-03-24T21:05:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "48c07a8e-d4ef-4ef1-b2fe-e90b97bbdf7a", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
93222026-03-24T21:05:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
93232026-03-24T21:05:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "f87db1a2-60bc-4a86-846c-67771e20b6ae", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
93242026-03-24T21:05:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
93252026-03-24T21:05:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "550ac5ab-7a26-4e0f-9bed-665644e97313", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
93262026-03-24T21:05:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
93272026-03-24T21:05:05.526ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
93282026-03-24T21:05:05.526ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 5.342854ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
93292026-03-24T21:05:05.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
93302026-03-24T21:05:05.526ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 740297637 } })
93312026-03-24T21:05:05.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93322026-03-24T21:05:05.527ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
93332026-03-24T21:05:05.527ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 6.553537ms iteration = 17 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
93342026-03-24T21:05:05.527ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93352026-03-24T21:05:05.528ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
93362026-03-24T21:05:05.528ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
93372026-03-24T21:05:05.529ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "023fce1a-1d29-45a2-b680-d77bf1c6cb15", "content-length": "26", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} })
93382026-03-24T21:05:05.529ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
93392026-03-24T21:05:05.529ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
93402026-03-24T21:05:05.529ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 743176715 } })
93412026-03-24T21:05:05.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
93422026-03-24T21:05:05.530ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93432026-03-24T21:05:05.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
93442026-03-24T21:05:05.532ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 10.977156ms iteration = 17 status = {}
93452026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
93462026-03-24T21:05:05.533ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2026-03-24 20:55:05.521072670 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
93472026-03-24T21:05:05.533ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
93482026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
93492026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.137857ms iteration = 9 status = {"expiration":"2026-03-24T20:55:05.521072670Z","pruned":[]}
93502026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
93512026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
93522026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
93532026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93542026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
93552026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
93562026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
93572026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 43 reason = Timeout
93582026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
93592026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
93602026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
93612026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
93622026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 97 reason = Timeout
93632026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
93642026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 22 reason = Timeout
93652026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93662026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
93672026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
93682026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93692026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
93702026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
93712026-03-24T21:05:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
93722026-03-24T21:05:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
93732026-03-24T21:05:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93742026-03-24T21:05:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
93752026-03-24T21:05:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
93762026-03-24T21:05:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
93772026-03-24T21:05:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
93782026-03-24T21:05:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
93792026-03-24T21:05:05.535ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
93802026-03-24T21:05:05.535ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
93812026-03-24T21:05:05.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
93822026-03-24T21:05:05.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
93832026-03-24T21:05:05.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93842026-03-24T21:05:05.536ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
93852026-03-24T21:05:05.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93862026-03-24T21:05:05.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
93872026-03-24T21:05:05.538ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 17.882982ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
93882026-03-24T21:05:05.539ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93892026-03-24T21:05:05.540ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93902026-03-24T21:05:05.541ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
93912026-03-24T21:05:05.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93922026-03-24T21:05:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
93932026-03-24T21:05:05.541ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
93942026-03-24T21:05:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93952026-03-24T21:05:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
93962026-03-24T21:05:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
93972026-03-24T21:05:05.544ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
93982026-03-24T21:05:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
93992026-03-24T21:05:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
94002026-03-24T21:05:05.544ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
94012026-03-24T21:05:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
94022026-03-24T21:05:05.547ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
94032026-03-24T21:05:05.547ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
94042026-03-24T21:05:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
94052026-03-24T21:05:05.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94062026-03-24T21:05:05.552ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
94072026-03-24T21:05:05.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94082026-03-24T21:05:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94092026-03-24T21:05:05.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
94102026-03-24T21:05:05.554ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
94112026-03-24T21:05:05.554ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
94122026-03-24T21:05:05.554ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
94132026-03-24T21:05:05.554ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
94142026-03-24T21:05:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
94152026-03-24T21:05:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
94162026-03-24T21:05:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
94172026-03-24T21:05:05.555ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 769140035 } })
94182026-03-24T21:05:05.556ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
94192026-03-24T21:05:05.556ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 769961937 } })
94202026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 20 reason = Timeout
94212026-03-24T21:05:05.558ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
94222026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 9 reason = Timeout
94232026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
94242026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 21.771\u{b5}s iteration = 20 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
94252026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 17 reason = Timeout
94262026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
94272026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94282026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 9 reason = Timeout
94292026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
94302026-03-24T21:05:05.558ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
94312026-03-24T21:05:05.558ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
94322026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
94332026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.935806ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
94342026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
94352026-03-24T21:05:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
94362026-03-24T21:05:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94372026-03-24T21:05:05.559ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7afbdab4-41b1-4499-968f-2a8016858e43", "content-length": "26", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} })
94382026-03-24T21:05:05.559ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
94392026-03-24T21:05:05.559ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
94402026-03-24T21:05:05.559ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
94412026-03-24T21:05:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
94422026-03-24T21:05:05.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
94432026-03-24T21:05:05.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
94442026-03-24T21:05:05.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
94452026-03-24T21:05:05.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
94462026-03-24T21:05:05.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
94472026-03-24T21:05:05.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
94482026-03-24T21:05:05.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
94492026-03-24T21:05:05.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
94502026-03-24T21:05:05.560ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
94512026-03-24T21:05:05.560ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
94522026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
94532026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
94542026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
94552026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
94562026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
94572026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
94582026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
94592026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
94602026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
94612026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94622026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
94632026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94642026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
94652026-03-24T21:05:05.561ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 774776197 } })
94662026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
94672026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
94682026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
94692026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
94702026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
94712026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
94722026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
94732026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
94742026-03-24T21:05:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
94752026-03-24T21:05:05.562ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
94762026-03-24T21:05:05.562ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 3.593327ms iteration = 9 status = {"by_rx":{},"error":null}
94772026-03-24T21:05:05.564ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
94782026-03-24T21:05:05.564ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94792026-03-24T21:05:05.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94802026-03-24T21:05:05.564ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 778098777 } })
94812026-03-24T21:05:05.565ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94822026-03-24T21:05:05.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
94832026-03-24T21:05:05.566ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
94842026-03-24T21:05:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
94852026-03-24T21:05:05.568ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 7.354158ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
94862026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): activating background_task = trust_quorum_manager iteration = 9 reason = Timeout
94872026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 17 reason = Timeout
94882026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
94892026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 16.87\u{b5}s iteration = 17 status = {"disabled":true,"errors":[],"sps":[]}
94902026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 33 reason = Timeout
94912026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): activating background_task = attached_subnet_manager iteration = 9 reason = Timeout
94922026-03-24T21:05:05.577ZINFOnexus (BackgroundTasks): starting attached subnet manager work background_task = attached_subnet_manager file = nexus/src/app/background/tasks/attached_subnets.rs:374
94932026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
94942026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 33 reason = Timeout
94952026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
94962026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 9 reason = Timeout
94972026-03-24T21:05:05.577ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
94982026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
94992026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
95002026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
95012026-03-24T21:05:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
95022026-03-24T21:05:05.578ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1291, tv_nsec: 791372055 } })
95032026-03-24T21:05:05.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
95042026-03-24T21:05:05.580ZDEBGnexus (BackgroundTasks): roles background_task = attached_subnet_manager roles = RoleSet { roles: {} }
95052026-03-24T21:05:05.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = attached_subnet_manager resource = Database result = Ok(())
95062026-03-24T21:05:05.635ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95072026-03-24T21:05:05.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
95082026-03-24T21:05:05.641ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
95092026-03-24T21:05:05.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
95102026-03-24T21:05:05.648ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95112026-03-24T21:05:05.650ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95122026-03-24T21:05:05.652ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
95132026-03-24T21:05:05.652ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95142026-03-24T21:05:05.652ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
95152026-03-24T21:05:05.652ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
95162026-03-24T21:05:05.652ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
95172026-03-24T21:05:05.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
95182026-03-24T21:05:05.653ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95192026-03-24T21:05:05.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
95202026-03-24T21:05:05.654ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
95212026-03-24T21:05:05.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
95222026-03-24T21:05:05.654ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
95232026-03-24T21:05:05.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
95242026-03-24T21:05:05.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
95252026-03-24T21:05:05.655ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95262026-03-24T21:05:05.657ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95272026-03-24T21:05:05.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95282026-03-24T21:05:05.659ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
95292026-03-24T21:05:05.659ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
95302026-03-24T21:05:05.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
95312026-03-24T21:05:05.660ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
95322026-03-24T21:05:05.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
95332026-03-24T21:05:05.661ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
95342026-03-24T21:05:05.661ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95352026-03-24T21:05:05.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
95362026-03-24T21:05:05.661ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
95372026-03-24T21:05:05.661ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
95382026-03-24T21:05:05.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
95392026-03-24T21:05:05.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
95402026-03-24T21:05:05.661ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
95412026-03-24T21:05:05.661ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
95422026-03-24T21:05:05.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
95432026-03-24T21:05:05.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
95442026-03-24T21:05:05.662ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
95452026-03-24T21:05:05.662ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
95462026-03-24T21:05:05.662ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
95472026-03-24T21:05:05.662ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
95482026-03-24T21:05:05.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
95492026-03-24T21:05:05.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
95502026-03-24T21:05:05.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
95512026-03-24T21:05:05.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
95522026-03-24T21:05:05.663ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95532026-03-24T21:05:05.663ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95542026-03-24T21:05:05.664ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95552026-03-24T21:05:05.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
95562026-03-24T21:05:05.664ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95572026-03-24T21:05:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
95582026-03-24T21:05:05.665ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95592026-03-24T21:05:05.666ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
95602026-03-24T21:05:05.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95612026-03-24T21:05:05.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95622026-03-24T21:05:05.667ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
95632026-03-24T21:05:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
95642026-03-24T21:05:05.667ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
95652026-03-24T21:05:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
95662026-03-24T21:05:05.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Fleet result = Ok(())
95672026-03-24T21:05:05.669ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
95682026-03-24T21:05:05.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
95692026-03-24T21:05:05.669ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 136.203753ms iteration = 17 status = 0
95702026-03-24T21:05:05.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
95712026-03-24T21:05:05.669ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
95722026-03-24T21:05:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
95732026-03-24T21:05:05.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
95742026-03-24T21:05:05.672ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
95752026-03-24T21:05:05.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
95762026-03-24T21:05:05.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
95772026-03-24T21:05:05.674ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
95782026-03-24T21:05:05.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
95792026-03-24T21:05:05.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
95802026-03-24T21:05:05.675ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 116.893992ms iteration = 17 status = {"errors":[],"requests_created_ok":[]}
95812026-03-24T21:05:05.675ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
95822026-03-24T21:05:05.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
95832026-03-24T21:05:05.677ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
95842026-03-24T21:05:05.677ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 143.808678ms iteration = 97 status = {"config_updated":false}
95852026-03-24T21:05:05.677ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = bfe73e65-b1ea-44ad-b413-8bbb37fec3ef original_time_started = 2026-03-24 21:04:09.231336 UTC
95862026-03-24T21:05:05.677ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 143.985772ms iteration = 43 status = {"Loaded":{"collection_id":"bfe73e65-b1ea-44ad-b413-8bbb37fec3ef","time_loaded":"2026-03-24T21:05:05.533316700Z","time_started":"2026-03-24T21:04:09.231336Z"}}
95872026-03-24T21:05:05.679ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
95882026-03-24T21:05:05.679ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
95892026-03-24T21:05:05.679ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 124.395044ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
95902026-03-24T21:05:05.680ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 122.277347ms iteration = 9 status = {"dispatched":[],"errors":[],"glob_version":"242.0.0","globs_reprocessed":{},"no_receivers":[]}
95912026-03-24T21:05:05.681ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95922026-03-24T21:05:05.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95932026-03-24T21:05:05.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
95942026-03-24T21:05:05.683ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
95952026-03-24T21:05:05.684ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95962026-03-24T21:05:05.684ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
95972026-03-24T21:05:05.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95982026-03-24T21:05:05.685ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95992026-03-24T21:05:05.687ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
96002026-03-24T21:05:05.687ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96012026-03-24T21:05:05.689ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 111.968229ms iteration = 33 status = "NoSitrep"
96022026-03-24T21:05:05.690ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96032026-03-24T21:05:05.690ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
96042026-03-24T21:05:05.690ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
96052026-03-24T21:05:05.690ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
96062026-03-24T21:05:05.690ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
96072026-03-24T21:05:05.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
96082026-03-24T21:05:05.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
96092026-03-24T21:05:05.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
96102026-03-24T21:05:05.691ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
96112026-03-24T21:05:05.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 131.008372ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
96122026-03-24T21:05:05.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
96132026-03-24T21:05:05.692ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
96142026-03-24T21:05:05.692ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
96152026-03-24T21:05:05.692ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
96162026-03-24T21:05:05.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
96172026-03-24T21:05:05.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96182026-03-24T21:05:05.693ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
96192026-03-24T21:05:05.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
96202026-03-24T21:05:05.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
96212026-03-24T21:05:05.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
96222026-03-24T21:05:05.694ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
96232026-03-24T21:05:05.694ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96242026-03-24T21:05:05.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96252026-03-24T21:05:05.695ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
96262026-03-24T21:05:05.696ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
96272026-03-24T21:05:05.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
96282026-03-24T21:05:05.697ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
96292026-03-24T21:05:05.701ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
96302026-03-24T21:05:05.704ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
96312026-03-24T21:05:05.704ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
96322026-03-24T21:05:05.705ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
96332026-03-24T21:05:05.705ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
96342026-03-24T21:05:05.705ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
96352026-03-24T21:05:05.705ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
96362026-03-24T21:05:05.705ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
96372026-03-24T21:05:05.705ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96382026-03-24T21:05:05.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96392026-03-24T21:05:05.712ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
96402026-03-24T21:05:05.716ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
96412026-03-24T21:05:05.720ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
96422026-03-24T21:05:05.722ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
96432026-03-24T21:05:05.725ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
96442026-03-24T21:05:05.725ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = SledList step id = Fetch
96452026-03-24T21:05:05.725ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96462026-03-24T21:05:05.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96472026-03-24T21:05:05.735ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
96482026-03-24T21:05:05.735ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
96492026-03-24T21:05:05.735ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
96502026-03-24T21:05:05.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
96512026-03-24T21:05:05.737ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96522026-03-24T21:05:05.739ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
96532026-03-24T21:05:05.741ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96542026-03-24T21:05:05.745ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
96552026-03-24T21:05:05.745ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96562026-03-24T21:05:05.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96572026-03-24T21:05:05.746ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96582026-03-24T21:05:05.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
96592026-03-24T21:05:05.747ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
96602026-03-24T21:05:05.747ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
96612026-03-24T21:05:05.747ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 169.913122ms iteration = 33 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
96622026-03-24T21:05:05.748ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 187.895396ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
96632026-03-24T21:05:05.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96642026-03-24T21:05:05.750ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
96652026-03-24T21:05:05.750ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
96662026-03-24T21:05:05.753ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
96672026-03-24T21:05:05.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96682026-03-24T21:05:05.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96692026-03-24T21:05:05.755ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
96702026-03-24T21:05:05.755ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
96712026-03-24T21:05:05.755ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe49ca42-574c-48e8-9ab7-301b01180a65", "content-length": "372", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} })
96722026-03-24T21:05:05.755ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96732026-03-24T21:05:05.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96742026-03-24T21:05:05.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96752026-03-24T21:05:05.759ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
96762026-03-24T21:05:05.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
96772026-03-24T21:05:05.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
96782026-03-24T21:05:05.761ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
96792026-03-24T21:05:05.761ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 228.613895ms iteration = 49 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
96802026-03-24T21:05:05.771ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
96812026-03-24T21:05:05.771ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
96822026-03-24T21:05:05.771ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 238.384048ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
96832026-03-24T21:05:05.780ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
96842026-03-24T21:05:05.780ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
96852026-03-24T21:05:05.781ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
96862026-03-24T21:05:05.781ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "19594e0c-e626-438f-bd9a-29dfc9e5f644", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} })
96872026-03-24T21:05:05.781ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96882026-03-24T21:05:05.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
96892026-03-24T21:05:05.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96902026-03-24T21:05:05.783ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
96912026-03-24T21:05:05.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
96922026-03-24T21:05:05.784ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96932026-03-24T21:05:05.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96942026-03-24T21:05:05.786ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 225.276265ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
96952026-03-24T21:05:05.786ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 239.533469ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
96962026-03-24T21:05:05.787ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
96972026-03-24T21:05:05.787ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 254.656637ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
96982026-03-24T21:05:05.788ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96992026-03-24T21:05:05.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97002026-03-24T21:05:05.788ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
97012026-03-24T21:05:05.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
97022026-03-24T21:05:05.790ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97032026-03-24T21:05:05.790ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
97042026-03-24T21:05:05.790ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
97052026-03-24T21:05:05.790ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
97062026-03-24T21:05:05.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97072026-03-24T21:05:05.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
97082026-03-24T21:05:05.791ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
97092026-03-24T21:05:05.791ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
97102026-03-24T21:05:05.791ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
97112026-03-24T21:05:05.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97122026-03-24T21:05:05.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
97132026-03-24T21:05:05.792ZINFOnexus (BackgroundTasks): Loaded 0 active trust quorum configurations from database background_task = trust_quorum_manager bgtask = TrustQuorumManager file = nexus/src/app/background/tasks/trust_quorum.rs:83
97142026-03-24T21:05:05.792ZDEBGnexus (BackgroundTasks): activation complete background_task = trust_quorum_manager elapsed = 214.928946ms iteration = 9 status = {"PerRackStatus":{"errors":[],"statuses":[]}}
97152026-03-24T21:05:05.792ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
97162026-03-24T21:05:05.792ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
97172026-03-24T21:05:05.793ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97182026-03-24T21:05:05.793ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97192026-03-24T21:05:05.793ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
97202026-03-24T21:05:05.793ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
97212026-03-24T21:05:05.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
97222026-03-24T21:05:05.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97232026-03-24T21:05:05.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97242026-03-24T21:05:05.794ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
97252026-03-24T21:05:05.794ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 233.955559ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
97262026-03-24T21:05:05.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97272026-03-24T21:05:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
97282026-03-24T21:05:05.799ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
97292026-03-24T21:05:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
97302026-03-24T21:05:05.799ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
97312026-03-24T21:05:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
97322026-03-24T21:05:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
97332026-03-24T21:05:05.800ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "cf3d02ef-23e3-42bd-86dd-8f3b8ca99583", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} })
97342026-03-24T21:05:05.800ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 239.95446ms iteration = 25 status = {}
97352026-03-24T21:05:05.800ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
97362026-03-24T21:05:05.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97372026-03-24T21:05:05.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97382026-03-24T21:05:05.802ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
97392026-03-24T21:05:05.802ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
97402026-03-24T21:05:05.802ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2026-03-24 20:57:07.125641 UTC dns_group = external generation = 2
97412026-03-24T21:05:05.802ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 281.297345ms iteration = 9 status = {"generation":2}
97422026-03-24T21:05:05.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97432026-03-24T21:05:05.803ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
97442026-03-24T21:05:05.803ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
97452026-03-24T21:05:05.803ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2026-03-24 20:57:05.047625 UTC dns_group = internal generation = 1
97462026-03-24T21:05:05.803ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 287.546783ms iteration = 9 status = {"generation":1}
97472026-03-24T21:05:05.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97482026-03-24T21:05:05.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97492026-03-24T21:05:05.808ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
97502026-03-24T21:05:05.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97512026-03-24T21:05:05.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97522026-03-24T21:05:05.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97532026-03-24T21:05:05.813ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
97542026-03-24T21:05:05.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97552026-03-24T21:05:05.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97562026-03-24T21:05:05.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97572026-03-24T21:05:05.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97582026-03-24T21:05:05.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97592026-03-24T21:05:05.821ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
97602026-03-24T21:05:05.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97612026-03-24T21:05:05.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97622026-03-24T21:05:05.822ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 261.625744ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
97632026-03-24T21:05:05.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97642026-03-24T21:05:05.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97652026-03-24T21:05:05.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97662026-03-24T21:05:05.836ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
97672026-03-24T21:05:05.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97682026-03-24T21:05:05.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97692026-03-24T21:05:05.838ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
97702026-03-24T21:05:05.838ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
97712026-03-24T21:05:05.838ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
97722026-03-24T21:05:05.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
97732026-03-24T21:05:05.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97742026-03-24T21:05:05.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97752026-03-24T21:05:05.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97762026-03-24T21:05:05.847ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
97772026-03-24T21:05:05.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97782026-03-24T21:05:05.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97792026-03-24T21:05:05.853ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97802026-03-24T21:05:05.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
97812026-03-24T21:05:05.857ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97822026-03-24T21:05:05.858ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
97832026-03-24T21:05:05.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
97842026-03-24T21:05:05.859ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
97852026-03-24T21:05:05.859ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97862026-03-24T21:05:05.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
97872026-03-24T21:05:05.863ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97882026-03-24T21:05:05.866ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
97892026-03-24T21:05:05.866ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97902026-03-24T21:05:05.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
97912026-03-24T21:05:05.881ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2026-03-24T20:56:59.957750Z, time_modified: 2026-03-24T20:56:59.957750Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }) }
97922026-03-24T21:05:05.881ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 360.09289ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]}},"default_endpoint":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
97932026-03-24T21:05:05.884ZDEBGnexus (BackgroundTasks): listed attached subnets background_task = attached_subnet_manager n_subnets = 0
97942026-03-24T21:05:05.884ZDEBGnexus (BackgroundTasks): activation complete background_task = attached_subnet_manager elapsed = 307.285726ms iteration = 9 status = {"db_error":null,"dendrite":{"switch0":{"errors":[],"n_subnets_added":0,"n_subnets_removed":0,"n_total_subnets":0}},"sled":{}}
97952026-03-24T21:05:05.885ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
97962026-03-24T21:05:05.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
97972026-03-24T21:05:05.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
97982026-03-24T21:05:05.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
97992026-03-24T21:05:05.889ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98002026-03-24T21:05:05.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98012026-03-24T21:05:05.891ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
98022026-03-24T21:05:05.891ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
98032026-03-24T21:05:05.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
98042026-03-24T21:05:05.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98052026-03-24T21:05:05.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98062026-03-24T21:05:05.897ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
98072026-03-24T21:05:05.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98082026-03-24T21:05:05.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98092026-03-24T21:05:05.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98102026-03-24T21:05:05.902ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
98112026-03-24T21:05:05.903ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "d15d45a3-50f2-41db-979c-cc7d9da9c463", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} })
98122026-03-24T21:05:05.903ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
98132026-03-24T21:05:05.903ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
98142026-03-24T21:05:05.903ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 325.761454ms iteration = 9 status = {"errors":[],"probes_by_sled":{"b4293d4f-3308-4dfa-b261-a103215913e4":0}}
98152026-03-24T21:05:05.903ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Dependency
98162026-03-24T21:05:05.903ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
98172026-03-24T21:05:05.903ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98182026-03-24T21:05:05.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98192026-03-24T21:05:05.903ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
98202026-03-24T21:05:05.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98212026-03-24T21:05:05.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98222026-03-24T21:05:05.906ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98232026-03-24T21:05:05.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98242026-03-24T21:05:05.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98252026-03-24T21:05:05.908ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
98262026-03-24T21:05:05.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98272026-03-24T21:05:05.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98282026-03-24T21:05:05.909ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98292026-03-24T21:05:05.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98302026-03-24T21:05:05.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98312026-03-24T21:05:05.911ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
98322026-03-24T21:05:05.911ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98332026-03-24T21:05:05.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98342026-03-24T21:05:05.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98352026-03-24T21:05:05.912ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
98362026-03-24T21:05:05.912ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
98372026-03-24T21:05:05.912ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 351.762225ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
98382026-03-24T21:05:05.915ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
98392026-03-24T21:05:05.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98402026-03-24T21:05:05.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98412026-03-24T21:05:05.916ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
98422026-03-24T21:05:05.916ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
98432026-03-24T21:05:05.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98442026-03-24T21:05:05.920ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
98452026-03-24T21:05:05.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98462026-03-24T21:05:05.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98472026-03-24T21:05:05.920ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e021e6f1-5228-4aaa-89d4-9f7881f99197", "content-length": "372", "date": "Tue, 24 Mar 2026 21:05:05 GMT"} })
98482026-03-24T21:05:05.920ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98492026-03-24T21:05:05.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98502026-03-24T21:05:05.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98512026-03-24T21:05:05.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98522026-03-24T21:05:05.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98532026-03-24T21:05:05.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98542026-03-24T21:05:05.927ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:118 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
98552026-03-24T21:05:05.927ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 366.947125ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
98562026-03-24T21:05:05.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98572026-03-24T21:05:05.930ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 397.54981ms iteration = 28 status = {"physical_disks_added":0}
98582026-03-24T21:05:05.933ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
98592026-03-24T21:05:05.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98602026-03-24T21:05:05.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98612026-03-24T21:05:05.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98622026-03-24T21:05:05.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98632026-03-24T21:05:05.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98642026-03-24T21:05:05.942ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
98652026-03-24T21:05:05.945ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = DeployNexusRecords step id = Ensure
98662026-03-24T21:05:05.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
98672026-03-24T21:05:05.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
98682026-03-24T21:05:05.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98692026-03-24T21:05:05.949ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
98702026-03-24T21:05:05.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
98712026-03-24T21:05:05.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
98722026-03-24T21:05:05.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
98732026-03-24T21:05:05.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
98742026-03-24T21:05:05.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98752026-03-24T21:05:05.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
98762026-03-24T21:05:05.998ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
98772026-03-24T21:05:05.998ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
98782026-03-24T21:05:06.004ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "85f8e5b9-f312-4613-93a7-c30069b6d062", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
98792026-03-24T21:05:06.004ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98802026-03-24T21:05:06.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98812026-03-24T21:05:06.006ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
98822026-03-24T21:05:06.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
98832026-03-24T21:05:06.008ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98842026-03-24T21:05:06.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98852026-03-24T21:05:06.011ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98862026-03-24T21:05:06.011ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
98872026-03-24T21:05:06.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98882026-03-24T21:05:06.011ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0ec8bd1-558b-4ce7-a5e0-4c43f648d050", "content-length": "691", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
98892026-03-24T21:05:06.011ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
98902026-03-24T21:05:06.011ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
98912026-03-24T21:05:06.011ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
98922026-03-24T21:05:06.011ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 478.747809ms iteration = 17 status = {}
98932026-03-24T21:05:06.014ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
98942026-03-24T21:05:06.014ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
98952026-03-24T21:05:06.015ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d112f193-5d6c-46e1-872b-c72d5befb92d", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
98962026-03-24T21:05:06.015ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 112.144424ms iteration = 26 status = {}
98972026-03-24T21:05:06.018ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = SledAgent step id = Ensure
98982026-03-24T21:05:06.020ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
98992026-03-24T21:05:06.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99002026-03-24T21:05:06.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99012026-03-24T21:05:06.025ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99022026-03-24T21:05:06.025ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "71b612e7-6a18-411b-85ed-b1df1bc746bb", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} }) sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
99032026-03-24T21:05:06.025ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = FirewallRules step id = Ensure
99042026-03-24T21:05:06.025ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99052026-03-24T21:05:06.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99062026-03-24T21:05:06.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99072026-03-24T21:05:06.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99082026-03-24T21:05:06.028ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
99092026-03-24T21:05:06.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99102026-03-24T21:05:06.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99112026-03-24T21:05:06.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99122026-03-24T21:05:06.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99132026-03-24T21:05:06.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99142026-03-24T21:05:06.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99152026-03-24T21:05:06.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99162026-03-24T21:05:06.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99172026-03-24T21:05:06.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99182026-03-24T21:05:06.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99192026-03-24T21:05:06.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99202026-03-24T21:05:06.036ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
99212026-03-24T21:05:06.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99222026-03-24T21:05:06.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99232026-03-24T21:05:06.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99242026-03-24T21:05:06.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99252026-03-24T21:05:06.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99262026-03-24T21:05:06.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99272026-03-24T21:05:06.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99282026-03-24T21:05:06.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99292026-03-24T21:05:06.044ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
99302026-03-24T21:05:06.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99312026-03-24T21:05:06.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99322026-03-24T21:05:06.045ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
99332026-03-24T21:05:06.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99342026-03-24T21:05:06.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99352026-03-24T21:05:06.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99362026-03-24T21:05:06.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99372026-03-24T21:05:06.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99382026-03-24T21:05:06.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99392026-03-24T21:05:06.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99402026-03-24T21:05:06.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99412026-03-24T21:05:06.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99422026-03-24T21:05:06.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99432026-03-24T21:05:06.052ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
99442026-03-24T21:05:06.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99452026-03-24T21:05:06.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99462026-03-24T21:05:06.054ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
99472026-03-24T21:05:06.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99482026-03-24T21:05:06.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99492026-03-24T21:05:06.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99502026-03-24T21:05:06.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99512026-03-24T21:05:06.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99522026-03-24T21:05:06.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99532026-03-24T21:05:06.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99542026-03-24T21:05:06.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99552026-03-24T21:05:06.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99562026-03-24T21:05:06.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99572026-03-24T21:05:06.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99582026-03-24T21:05:06.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99592026-03-24T21:05:06.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99602026-03-24T21:05:06.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99612026-03-24T21:05:06.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99622026-03-24T21:05:06.072ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
99632026-03-24T21:05:06.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99642026-03-24T21:05:06.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99652026-03-24T21:05:06.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99662026-03-24T21:05:06.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99672026-03-24T21:05:06.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99682026-03-24T21:05:06.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99692026-03-24T21:05:06.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99702026-03-24T21:05:06.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99712026-03-24T21:05:06.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99722026-03-24T21:05:06.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99732026-03-24T21:05:06.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99742026-03-24T21:05:06.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99752026-03-24T21:05:06.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99762026-03-24T21:05:06.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99772026-03-24T21:05:06.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99782026-03-24T21:05:06.087ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "40235ac2-c539-4244-b206-630f7dc68d65", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
99792026-03-24T21:05:06.087ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
99802026-03-24T21:05:06.087ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
99812026-03-24T21:05:06.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
99822026-03-24T21:05:06.091ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
99832026-03-24T21:05:06.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99842026-03-24T21:05:06.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99852026-03-24T21:05:06.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99862026-03-24T21:05:06.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99872026-03-24T21:05:06.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99882026-03-24T21:05:06.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99892026-03-24T21:05:06.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99902026-03-24T21:05:06.102ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
99912026-03-24T21:05:06.102ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99922026-03-24T21:05:06.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99932026-03-24T21:05:06.105ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
99942026-03-24T21:05:06.105ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
99952026-03-24T21:05:06.105ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = a88ba378-bdfe-4fa4-be4f-ce39bdb03651 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
99962026-03-24T21:05:06.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
99972026-03-24T21:05:06.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
99982026-03-24T21:05:06.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99992026-03-24T21:05:06.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100002026-03-24T21:05:06.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100012026-03-24T21:05:06.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100022026-03-24T21:05:06.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100032026-03-24T21:05:06.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100042026-03-24T21:05:06.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100052026-03-24T21:05:06.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100062026-03-24T21:05:06.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100072026-03-24T21:05:06.115ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
100082026-03-24T21:05:06.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100092026-03-24T21:05:06.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100102026-03-24T21:05:06.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100112026-03-24T21:05:06.118ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
100122026-03-24T21:05:06.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100132026-03-24T21:05:06.118ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
100142026-03-24T21:05:06.118ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 585.288652ms iteration = 22 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
100152026-03-24T21:05:06.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100162026-03-24T21:05:06.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100172026-03-24T21:05:06.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100182026-03-24T21:05:06.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100192026-03-24T21:05:06.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100202026-03-24T21:05:06.129ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
100212026-03-24T21:05:06.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100222026-03-24T21:05:06.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100232026-03-24T21:05:06.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100242026-03-24T21:05:06.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100252026-03-24T21:05:06.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100262026-03-24T21:05:06.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100272026-03-24T21:05:06.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100282026-03-24T21:05:06.138ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
100292026-03-24T21:05:06.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100302026-03-24T21:05:06.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100312026-03-24T21:05:06.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
100322026-03-24T21:05:06.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
100332026-03-24T21:05:06.144ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
100342026-03-24T21:05:06.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
100352026-03-24T21:05:06.146ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
100362026-03-24T21:05:06.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100372026-03-24T21:05:06.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100382026-03-24T21:05:06.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
100392026-03-24T21:05:06.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
100402026-03-24T21:05:06.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100412026-03-24T21:05:06.148ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 614.847769ms iteration = 17 status = {}
100422026-03-24T21:05:06.149ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
100432026-03-24T21:05:06.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100442026-03-24T21:05:06.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100452026-03-24T21:05:06.151ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
100462026-03-24T21:05:06.151ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
100472026-03-24T21:05:06.151ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
100482026-03-24T21:05:06.222ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
100492026-03-24T21:05:06.222ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
100502026-03-24T21:05:06.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100512026-03-24T21:05:06.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100522026-03-24T21:05:06.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100532026-03-24T21:05:06.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100542026-03-24T21:05:06.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100552026-03-24T21:05:06.230ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
100562026-03-24T21:05:06.230ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
100572026-03-24T21:05:06.231ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "d45c4f88-8461-4ea2-9fa0-510372bfb814", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
100582026-03-24T21:05:06.231ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
100592026-03-24T21:05:06.231ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = Dns step id = Ensure
100602026-03-24T21:05:06.231ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
100612026-03-24T21:05:06.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100622026-03-24T21:05:06.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100632026-03-24T21:05:06.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100642026-03-24T21:05:06.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100652026-03-24T21:05:06.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100662026-03-24T21:05:06.237ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
100672026-03-24T21:05:06.289ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
100682026-03-24T21:05:06.289ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
100692026-03-24T21:05:06.289ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
100702026-03-24T21:05:06.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100712026-03-24T21:05:06.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100722026-03-24T21:05:06.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100732026-03-24T21:05:06.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
100742026-03-24T21:05:06.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100752026-03-24T21:05:06.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100762026-03-24T21:05:06.337ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
100772026-03-24T21:05:06.337ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
100782026-03-24T21:05:06.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100792026-03-24T21:05:06.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100802026-03-24T21:05:06.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100812026-03-24T21:05:06.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
100822026-03-24T21:05:06.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100832026-03-24T21:05:06.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100842026-03-24T21:05:06.344ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
100852026-03-24T21:05:06.344ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
100862026-03-24T21:05:06.344ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
100872026-03-24T21:05:06.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100882026-03-24T21:05:06.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100892026-03-24T21:05:06.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100902026-03-24T21:05:06.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100912026-03-24T21:05:06.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100922026-03-24T21:05:06.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
100932026-03-24T21:05:06.351ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
100942026-03-24T21:05:06.351ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
100952026-03-24T21:05:06.351ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
100962026-03-24T21:05:06.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100972026-03-24T21:05:06.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100982026-03-24T21:05:06.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100992026-03-24T21:05:06.357ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
101002026-03-24T21:05:06.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101012026-03-24T21:05:06.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101022026-03-24T21:05:06.360ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
101032026-03-24T21:05:06.360ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
101042026-03-24T21:05:06.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101052026-03-24T21:05:06.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101062026-03-24T21:05:06.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101072026-03-24T21:05:06.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
101082026-03-24T21:05:06.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101092026-03-24T21:05:06.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101102026-03-24T21:05:06.366ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
101112026-03-24T21:05:06.366ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
101122026-03-24T21:05:06.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101132026-03-24T21:05:06.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101142026-03-24T21:05:06.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101152026-03-24T21:05:06.369ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
101162026-03-24T21:05:06.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101172026-03-24T21:05:06.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101182026-03-24T21:05:06.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101192026-03-24T21:05:06.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101202026-03-24T21:05:06.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101212026-03-24T21:05:06.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101222026-03-24T21:05:06.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101232026-03-24T21:05:06.377ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
101242026-03-24T21:05:06.379ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
101252026-03-24T21:05:06.379ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
101262026-03-24T21:05:06.379ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = OmicronZones step id = Cleanup
101272026-03-24T21:05:06.379ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = OmicronZones step id = Cleanup
101282026-03-24T21:05:06.379ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = PhysicalDisks step id = Cleanup
101292026-03-24T21:05:06.379ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = Clickhouse step id = Ensure
101302026-03-24T21:05:06.379ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = Clickhouse step id = Ensure
101312026-03-24T21:05:06.380ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
101322026-03-24T21:05:06.384ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "5dfc3638-e401-49ed-954e-78249ad2568a", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
101332026-03-24T21:05:06.384ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = SupportBundles step id = Cleanup
101342026-03-24T21:05:06.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101352026-03-24T21:05:06.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101362026-03-24T21:05:06.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101372026-03-24T21:05:06.388ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
101382026-03-24T21:05:06.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101392026-03-24T21:05:06.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101402026-03-24T21:05:06.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101412026-03-24T21:05:06.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101422026-03-24T21:05:06.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101432026-03-24T21:05:06.398ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
101442026-03-24T21:05:06.444ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = OmicronZones step id = Cleanup
101452026-03-24T21:05:06.444ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
101462026-03-24T21:05:06.444ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
101472026-03-24T21:05:06.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101482026-03-24T21:05:06.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101492026-03-24T21:05:06.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101502026-03-24T21:05:06.447ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
101512026-03-24T21:05:06.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101522026-03-24T21:05:06.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101532026-03-24T21:05:06.501ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
101542026-03-24T21:05:06.501ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9e1c09c1-3960-4ecf-a904-19e137699ab3 (blueprint), false)
101552026-03-24T21:05:06.501ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = Cockroach step id = Ensure
101562026-03-24T21:05:06.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
101572026-03-24T21:05:06.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
101582026-03-24T21:05:06.523ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
101592026-03-24T21:05:06.523ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = d1c0dccc-d91f-42d3-b624-9f3216473a64 step component = MgsUpdates step id = Ensure
101602026-03-24T21:05:06.523ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Dependency
101612026-03-24T21:05:06.523ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 11 reason = Dependency
101622026-03-24T21:05:06.523ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101632026-03-24T21:05:06.523ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101642026-03-24T21:05:06.523ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.001979018s iteration = 10 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":2410,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":203995561,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":203995511,"secs":0}},"event_index":1,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":204006511,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":219439718,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":219439687,"secs":0}},"event_index":2,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":423464769,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":73540853,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":73540803,"secs":0}},"event_index":3,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":497021042,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6883726,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":6883686,"secs":0}},"event_index":4,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":503921188,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":205862171,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":205862121,"secs":0}},"event_index":5,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":709815700,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":147465757,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":147465717,"secs":0}},"event_index":6,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":857290127,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4930,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4890,"secs":0}},"event_index":7,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":857304557,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2680,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2650,"secs":0}},"event_index":8,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":857309077,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3310,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3280,"secs":0}},"event_index":9,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":857313687,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1590,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1560,"secs":0}},"event_index":10,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":857316517,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5786045,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":5786005,"secs":0}},"event_index":11,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":863103523,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":59596647,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":59596596,"secs":0}},"event_index":12,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":922704779,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":57014327,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":57014287,"secs":0}},"event_index":13,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":979741577,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":21842789,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":21842759,"secs":0}},"event_index":14,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":1602687,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5170,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5130,"secs":0}},"event_index":15,"execution_id":"38572900-a25d-4b01-891c-e39def9b09b8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":1617638,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3"}
101652026-03-24T21:05:06.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
101662026-03-24T21:05:06.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101672026-03-24T21:05:06.525ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101682026-03-24T21:05:06.525ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101692026-03-24T21:05:06.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
101702026-03-24T21:05:06.528ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101712026-03-24T21:05:06.528ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
101722026-03-24T21:05:06.528ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101732026-03-24T21:05:06.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101742026-03-24T21:05:06.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
101752026-03-24T21:05:06.569ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101762026-03-24T21:05:06.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101772026-03-24T21:05:06.571ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101782026-03-24T21:05:06.573ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
101792026-03-24T21:05:06.573ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101802026-03-24T21:05:06.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101812026-03-24T21:05:06.575ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
101822026-03-24T21:05:06.575ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 52.175247ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
101832026-03-24T21:05:06.609ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(dbfdf733-53f6-4263-bd98-b247cf0b5509, 0), (bfcfe115-13a5-4646-a408-f5f6ef741c06, 0), (6c89cf74-b314-4c93-82df-5c49e44c369f, 0), (5e0b62e9-318c-4b6d-bdeb-a335b2a72afc, 0)] collection_id = dbfdf733-53f6-4263-bd98-b247cf0b5509
101842026-03-24T21:05:06.609ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101852026-03-24T21:05:06.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
101862026-03-24T21:05:06.795ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 26 transaction = inventory_delete_collection
101872026-03-24T21:05:06.859ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = inventory_delete_collection
101882026-03-24T21:05:06.859ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = dbfdf733-53f6-4263-bd98-b247cf0b5509 file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
101892026-03-24T21:05:06.859ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101902026-03-24T21:05:06.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
101912026-03-24T21:05:06.990ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(6c89cf74-b314-4c93-82df-5c49e44c369f, 0), (5e0b62e9-318c-4b6d-bdeb-a335b2a72afc, 0), (bfe73e65-b1ea-44ad-b413-8bbb37fec3ef, 0)]
101922026-03-24T21:05:06.990ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1293, tv_nsec: 204331074 } })
101932026-03-24T21:05:06.992ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1345db7a-abc1-4a4c-a49c-e6c56102e521.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1920db37-87f5-4a96-b3ef-1db2eb59e9a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6b02d2eb-4373-4f12-8efc-4c0e3f4fa77b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("709c4704-f768-4640-a198-a23621ed6e11.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1293, tv_nsec: 206233265 } })
101942026-03-24T21:05:06.993ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
101952026-03-24T21:05:06.993ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
101962026-03-24T21:05:06.993ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
101972026-03-24T21:05:06.994ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e80c30e-65f7-4cba-b090-8d65497ed428", "content-length": "476", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
101982026-03-24T21:05:06.994ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
101992026-03-24T21:05:06.994ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "36a79f12-2ae7-4859-812f-3bf4ad882cef", "content-length": "734", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102002026-03-24T21:05:06.994ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
102012026-03-24T21:05:06.994ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5796c176-f953-4115-9357-9c69670238ff", "content-length": "111", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102022026-03-24T21:05:06.994ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
102032026-03-24T21:05:06.995ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec7a1907-8952-4ef6-9ff8-8f7edf180e2f", "content-length": "111", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102042026-03-24T21:05:06.995ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
102052026-03-24T21:05:06.995ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2458ee3d-271b-4ff8-aadf-52a3e8d69d31", "content-length": "163", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102062026-03-24T21:05:06.996ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
102072026-03-24T21:05:06.996ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5fb1652-1667-43c0-85fc-bdfe048c4d3c", "content-length": "163", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102082026-03-24T21:05:06.996ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
102092026-03-24T21:05:06.997ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a83237eb-065b-4749-ae1e-aabf12aa780c", "content-length": "166", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102102026-03-24T21:05:06.997ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
102112026-03-24T21:05:06.997ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4d9a7be-ea45-4a38-a96d-8434b890795a", "content-length": "166", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102122026-03-24T21:05:06.997ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
102132026-03-24T21:05:06.997ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b644816-5b5b-479d-8e82-0cc5fbddbd8b", "content-length": "702", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102142026-03-24T21:05:06.997ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
102152026-03-24T21:05:06.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dab9b1d-3fe6-4025-86e9-ee6a5c0688e6", "content-length": "727", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102162026-03-24T21:05:06.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
102172026-03-24T21:05:06.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2596f0eb-29bc-47a4-a3d8-6b03ce529fac", "content-length": "729", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102182026-03-24T21:05:06.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
102192026-03-24T21:05:06.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6aafee9-87a5-4b22-b1fc-c4ce378de035", "content-length": "728", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102202026-03-24T21:05:06.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
102212026-03-24T21:05:06.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "921b6f55-a8a9-4e86-b60a-5b4a8b27c1c9", "content-length": "722", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102222026-03-24T21:05:06.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
102232026-03-24T21:05:06.999ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3f0e136-3722-4afb-b7a1-bed76a0a4211", "content-length": "10", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102242026-03-24T21:05:06.999ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
102252026-03-24T21:05:06.999ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d10f2f5a-c3ad-43fc-beab-0e90d85eb8b2", "content-length": "144", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102262026-03-24T21:05:06.999ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
102272026-03-24T21:05:06.999ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9318854-6187-483d-9033-103280cdcab6", "content-length": "147", "date": "Tue, 24 Mar 2026 21:05:06 GMT"} })
102282026-03-24T21:05:06.999ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
102292026-03-24T21:05:07.000ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3c8cf43-ed20-4620-ac0e-22f76c2ff551", "content-length": "109", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102302026-03-24T21:05:07.000ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
102312026-03-24T21:05:07.000ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8eb7fa79-ef1d-4efe-a579-4b562be66bcc", "content-length": "109", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102322026-03-24T21:05:07.000ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
102332026-03-24T21:05:07.001ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "365499f4-c74a-4c63-93ca-c147550afe43", "content-length": "163", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102342026-03-24T21:05:07.001ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
102352026-03-24T21:05:07.001ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a481cb1-a830-44e8-9460-825c6176bbe9", "content-length": "163", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102362026-03-24T21:05:07.001ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
102372026-03-24T21:05:07.002ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec662447-21fc-4d63-ac62-945bdde2f627", "content-length": "166", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102382026-03-24T21:05:07.002ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
102392026-03-24T21:05:07.002ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f9e8a85-4c06-4eea-9e6b-1abbf53db5e5", "content-length": "166", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102402026-03-24T21:05:07.002ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
102412026-03-24T21:05:07.002ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6009effd-e2b9-4dc2-987e-7f43d9138086", "content-length": "702", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102422026-03-24T21:05:07.002ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
102432026-03-24T21:05:07.003ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a7efe89-82dc-488a-9910-3cf819b5d4d2", "content-length": "727", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102442026-03-24T21:05:07.003ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
102452026-03-24T21:05:07.003ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8eb76258-c60e-4683-8f95-d33942f5e32f", "content-length": "729", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102462026-03-24T21:05:07.003ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
102472026-03-24T21:05:07.003ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9cdab66-b3c8-453e-a313-1cc6cad6e0af", "content-length": "728", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102482026-03-24T21:05:07.003ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
102492026-03-24T21:05:07.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
102502026-03-24T21:05:07.005ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102512026-03-24T21:05:07.007ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
102522026-03-24T21:05:07.007ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
102532026-03-24T21:05:07.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
102542026-03-24T21:05:07.009ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102552026-03-24T21:05:07.012ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
102562026-03-24T21:05:07.012ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
102572026-03-24T21:05:07.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
102582026-03-24T21:05:07.016ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
102592026-03-24T21:05:07.016ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
102602026-03-24T21:05:07.106ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a03f7da-a6a8-4adc-934b-dbda20d3ddde", "content-length": "110672", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102612026-03-24T21:05:07.107ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
102622026-03-24T21:05:07.107ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
102632026-03-24T21:05:07.107ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
102642026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
102652026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
102662026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
102672026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
102682026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
102692026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "840c24e0-5cf3-4e7f-865b-262beaaa2a84", "content-length": "64", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102702026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6af34d8-8b1b-4a5f-b94a-7ac0c3870ac7", "content-length": "64", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102712026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
102722026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
102732026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f53e0dd-e1e5-4858-a872-8d4c9ceeeaf5", "content-length": "64", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102742026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa3206db-c0aa-44a9-8617-b15b125a1871", "content-length": "64", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102752026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
102762026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
102772026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e96360c2-4405-4e6d-b306-ff846f95b701", "content-length": "64", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102782026-03-24T21:05:07.107ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
102792026-03-24T21:05:07.114ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a20e19d-56ae-4300-98de-93d8f021683f", "content-length": "398256", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102802026-03-24T21:05:07.115ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "5006ec7c-e5c3-4ca5-b15a-5007cf749cf6", "content-length": "399996", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102812026-03-24T21:05:07.115ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "207f5784-78e4-45a9-9cf9-78847363a06e", "content-length": "429169", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102822026-03-24T21:05:07.115ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "be59ddf1-e76c-4745-805e-9161e4c4398f", "content-length": "402728", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102832026-03-24T21:05:07.117ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f850f00-c383-4b9e-a6f2-a1ea3dce1af7", "content-length": "407895", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102842026-03-24T21:05:07.120ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
102852026-03-24T21:05:07.120ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
102862026-03-24T21:05:07.120ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
102872026-03-24T21:05:07.120ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
102882026-03-24T21:05:07.120ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
102892026-03-24T21:05:07.122ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 (omicron_zone)
102902026-03-24T21:05:07.122ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
102912026-03-24T21:05:07.126ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "cae1705e-f944-4582-be2f-9c64e7f82a93", "content-length": "128", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102922026-03-24T21:05:07.126ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
102932026-03-24T21:05:07.129ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
102942026-03-24T21:05:07.129ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
102952026-03-24T21:05:07.130ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "15434e07-05b6-4feb-8129-c0bd2398167f", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
102962026-03-24T21:05:07.130ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
102972026-03-24T21:05:07.130ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
102982026-03-24T21:05:07.130ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
102992026-03-24T21:05:07.131ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cff89ff-dc0b-49f5-bf56-c27f14eb5c3f", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
103002026-03-24T21:05:07.131ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
103012026-03-24T21:05:07.131ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
103022026-03-24T21:05:07.131ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
103032026-03-24T21:05:07.131ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dd09d39-8694-4fe3-a341-617c45224d3f", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:05:07 GMT"} })
103042026-03-24T21:05:07.131ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
103052026-03-24T21:05:07.131ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
103062026-03-24T21:05:07.131ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
103072026-03-24T21:05:07.132ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103082026-03-24T21:05:07.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
103092026-03-24T21:05:07.134ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103102026-03-24T21:05:07.136ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
103112026-03-24T21:05:07.136ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103122026-03-24T21:05:07.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
103132026-03-24T21:05:07.362ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 536e3610-cdc3-4254-98ba-07880214c42c file = nexus/db-queries/src/db/datastore/inventory.rs:1757
103142026-03-24T21:05:07.362ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 536e3610-cdc3-4254-98ba-07880214c42c time_started = 2026-03-24 21:05:06.993628 UTC
103152026-03-24T21:05:07.362ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 44 reason = Dependency
103162026-03-24T21:05:07.362ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
103172026-03-24T21:05:07.362ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 839.196849ms iteration = 11 status = {"collection_id":"536e3610-cdc3-4254-98ba-07880214c42c","time_done":"2026-03-24 21:05:07.131970 UTC","time_started":"2026-03-24 21:05:06.993628 UTC"}
103182026-03-24T21:05:07.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
103192026-03-24T21:05:07.365ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103202026-03-24T21:05:07.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
103212026-03-24T21:05:07.368ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
103222026-03-24T21:05:07.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
103232026-03-24T21:05:07.433ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
103242026-03-24T21:05:07.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
103252026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 115.416832ms iteration = 44 status = {"Loaded":{"collection_id":"536e3610-cdc3-4254-98ba-07880214c42c","time_loaded":"2026-03-24T21:05:07.362755499Z","time_started":"2026-03-24T21:05:06.993628Z"}}
103262026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 21 reason = Dependency
103272026-03-24T21:05:07.478ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
103282026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Dependency
103292026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 14 reason = Dependency
103302026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 23 reason = Dependency
103312026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 14.4\u{b5}s iteration = 21 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
103322026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
103332026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
103342026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103352026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
103362026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
103372026-03-24T21:05:07.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103382026-03-24T21:05:07.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103392026-03-24T21:05:07.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103402026-03-24T21:05:07.480ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103412026-03-24T21:05:07.482ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
103422026-03-24T21:05:07.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
103432026-03-24T21:05:07.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
103442026-03-24T21:05:07.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
103452026-03-24T21:05:07.482ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
103462026-03-24T21:05:07.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
103472026-03-24T21:05:07.483ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
103482026-03-24T21:05:07.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103492026-03-24T21:05:07.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103502026-03-24T21:05:07.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103512026-03-24T21:05:07.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103522026-03-24T21:05:07.485ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
103532026-03-24T21:05:07.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
103542026-03-24T21:05:07.486ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
103552026-03-24T21:05:07.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
103562026-03-24T21:05:07.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
103572026-03-24T21:05:07.486ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103582026-03-24T21:05:07.486ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
103592026-03-24T21:05:07.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103602026-03-24T21:05:07.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103612026-03-24T21:05:07.488ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
103622026-03-24T21:05:07.488ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
103632026-03-24T21:05:07.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
103642026-03-24T21:05:07.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103652026-03-24T21:05:07.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103662026-03-24T21:05:07.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103672026-03-24T21:05:07.492ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
103682026-03-24T21:05:07.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103692026-03-24T21:05:07.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103702026-03-24T21:05:07.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103712026-03-24T21:05:07.496ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
103722026-03-24T21:05:07.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103732026-03-24T21:05:07.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103742026-03-24T21:05:07.499ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
103752026-03-24T21:05:07.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103762026-03-24T21:05:07.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103772026-03-24T21:05:07.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103782026-03-24T21:05:07.503ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
103792026-03-24T21:05:07.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103802026-03-24T21:05:07.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103812026-03-24T21:05:07.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103822026-03-24T21:05:07.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
103832026-03-24T21:05:07.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103842026-03-24T21:05:07.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103852026-03-24T21:05:07.510ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
103862026-03-24T21:05:07.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103872026-03-24T21:05:07.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103882026-03-24T21:05:07.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103892026-03-24T21:05:07.515ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
103902026-03-24T21:05:07.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103912026-03-24T21:05:07.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103922026-03-24T21:05:07.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103932026-03-24T21:05:07.520ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
103942026-03-24T21:05:07.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
103952026-03-24T21:05:07.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
103962026-03-24T21:05:07.522ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
103972026-03-24T21:05:07.522ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 44.762387ms iteration = 14 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"536e3610-cdc3-4254-98ba-07880214c42c","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
103982026-03-24T21:05:07.550ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 72.261678ms iteration = 29 status = {"physical_disks_added":0}
103992026-03-24T21:05:07.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104002026-03-24T21:05:07.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104012026-03-24T21:05:07.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104022026-03-24T21:05:07.585ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
104032026-03-24T21:05:07.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104042026-03-24T21:05:07.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104052026-03-24T21:05:07.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104062026-03-24T21:05:07.588ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
104072026-03-24T21:05:07.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104082026-03-24T21:05:07.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104092026-03-24T21:05:07.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104102026-03-24T21:05:07.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104112026-03-24T21:05:07.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104122026-03-24T21:05:07.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104132026-03-24T21:05:07.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104142026-03-24T21:05:07.599ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104152026-03-24T21:05:07.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104162026-03-24T21:05:07.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104172026-03-24T21:05:07.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104182026-03-24T21:05:07.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104192026-03-24T21:05:07.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104202026-03-24T21:05:07.606ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
104212026-03-24T21:05:07.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104222026-03-24T21:05:07.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104232026-03-24T21:05:07.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104242026-03-24T21:05:07.610ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104252026-03-24T21:05:07.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104262026-03-24T21:05:07.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104272026-03-24T21:05:07.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104282026-03-24T21:05:07.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104292026-03-24T21:05:07.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104302026-03-24T21:05:07.617ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
104312026-03-24T21:05:07.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104322026-03-24T21:05:07.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104332026-03-24T21:05:07.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104342026-03-24T21:05:07.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104352026-03-24T21:05:07.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104362026-03-24T21:05:07.625ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
104372026-03-24T21:05:07.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104382026-03-24T21:05:07.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104392026-03-24T21:05:07.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104402026-03-24T21:05:07.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104412026-03-24T21:05:07.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104422026-03-24T21:05:07.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
104432026-03-24T21:05:07.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104442026-03-24T21:05:07.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104452026-03-24T21:05:07.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104462026-03-24T21:05:07.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104472026-03-24T21:05:07.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104482026-03-24T21:05:07.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104492026-03-24T21:05:07.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104502026-03-24T21:05:07.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104512026-03-24T21:05:07.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104522026-03-24T21:05:07.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104532026-03-24T21:05:07.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104542026-03-24T21:05:07.659ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104552026-03-24T21:05:07.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104562026-03-24T21:05:07.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104572026-03-24T21:05:07.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104582026-03-24T21:05:07.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104592026-03-24T21:05:07.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104602026-03-24T21:05:07.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104612026-03-24T21:05:07.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104622026-03-24T21:05:07.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104632026-03-24T21:05:07.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104642026-03-24T21:05:07.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104652026-03-24T21:05:07.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104662026-03-24T21:05:07.674ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104672026-03-24T21:05:07.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104682026-03-24T21:05:07.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104692026-03-24T21:05:07.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104702026-03-24T21:05:07.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104712026-03-24T21:05:07.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104722026-03-24T21:05:07.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104732026-03-24T21:05:07.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104742026-03-24T21:05:07.682ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
104752026-03-24T21:05:07.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104762026-03-24T21:05:07.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104772026-03-24T21:05:07.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104782026-03-24T21:05:07.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104792026-03-24T21:05:07.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104802026-03-24T21:05:07.689ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
104812026-03-24T21:05:07.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104822026-03-24T21:05:07.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104832026-03-24T21:05:07.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104842026-03-24T21:05:07.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104852026-03-24T21:05:07.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104862026-03-24T21:05:07.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
104872026-03-24T21:05:07.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104882026-03-24T21:05:07.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104892026-03-24T21:05:07.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104902026-03-24T21:05:07.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104912026-03-24T21:05:07.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104922026-03-24T21:05:07.704ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
104932026-03-24T21:05:07.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104942026-03-24T21:05:07.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104952026-03-24T21:05:07.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104962026-03-24T21:05:07.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104972026-03-24T21:05:07.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104982026-03-24T21:05:07.709ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
104992026-03-24T21:05:07.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105002026-03-24T21:05:07.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105012026-03-24T21:05:07.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105022026-03-24T21:05:07.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105032026-03-24T21:05:07.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105042026-03-24T21:05:07.714ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
105052026-03-24T21:05:07.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105062026-03-24T21:05:07.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105072026-03-24T21:05:07.758ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
105082026-03-24T21:05:07.758ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
105092026-03-24T21:05:07.758ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 3c68660a-06d8-43c1-a06b-3a24e513cb49 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
105102026-03-24T21:05:07.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105112026-03-24T21:05:07.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105122026-03-24T21:05:07.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105132026-03-24T21:05:07.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
105142026-03-24T21:05:07.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105152026-03-24T21:05:07.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105162026-03-24T21:05:07.765ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
105172026-03-24T21:05:07.765ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
105182026-03-24T21:05:07.765ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 287.750229ms iteration = 23 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
105192026-03-24T21:05:08.551ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = d2568406-9dc1-40e6-b16a-6c7a91cdbf62 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
105202026-03-24T21:05:08.551ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = d2568406-9dc1-40e6-b16a-6c7a91cdbf62 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
105212026-03-24T21:05:08.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = d2568406-9dc1-40e6-b16a-6c7a91cdbf62 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
105222026-03-24T21:05:08.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 3209 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = d2568406-9dc1-40e6-b16a-6c7a91cdbf62 response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
105232026-03-24T21:05:08.555ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = ee49b327-675d-4c21-bd47-17898677d156 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
105242026-03-24T21:05:08.555ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = ee49b327-675d-4c21-bd47-17898677d156 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
105252026-03-24T21:05:08.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = ee49b327-675d-4c21-bd47-17898677d156 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
105262026-03-24T21:05:08.558ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2754 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = ee49b327-675d-4c21-bd47-17898677d156 response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
105272026-03-24T21:05:08.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 399 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 76eb4394-74bc-41ca-8a03-42ec52365fb1 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
105282026-03-24T21:05:10.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 98 reason = Timeout
105292026-03-24T21:05:10.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105302026-03-24T21:05:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105312026-03-24T21:05:10.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105322026-03-24T21:05:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
105332026-03-24T21:05:10.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105342026-03-24T21:05:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105352026-03-24T21:05:10.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
105362026-03-24T21:05:10.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.765509ms iteration = 98 status = {"config_updated":false}
105372026-03-24T21:05:15.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
105382026-03-24T21:05:15.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105392026-03-24T21:05:15.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 99 reason = Timeout
105402026-03-24T21:05:15.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105412026-03-24T21:05:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105422026-03-24T21:05:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105432026-03-24T21:05:15.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105442026-03-24T21:05:15.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105452026-03-24T21:05:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
105462026-03-24T21:05:15.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105472026-03-24T21:05:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
105482026-03-24T21:05:15.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105492026-03-24T21:05:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105502026-03-24T21:05:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105512026-03-24T21:05:15.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
105522026-03-24T21:05:15.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.517943ms iteration = 99 status = {"config_updated":false}
105532026-03-24T21:05:15.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105542026-03-24T21:05:15.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105552026-03-24T21:05:15.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105562026-03-24T21:05:15.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
105572026-03-24T21:05:15.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
105582026-03-24T21:05:15.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
105592026-03-24T21:05:15.559ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
105602026-03-24T21:05:15.559ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.917629ms iteration = 50 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
105612026-03-24T21:05:15.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
105622026-03-24T21:05:15.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
105632026-03-24T21:05:15.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
105642026-03-24T21:05:15.601ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
105652026-03-24T21:05:15.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
105662026-03-24T21:05:15.641ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 80.445559ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
105672026-03-24T21:05:18.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 656 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = e376a84d-d25f-44ac-8eda-5f70eaf08716 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
105682026-03-24T21:05:20.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 45 reason = Timeout
105692026-03-24T21:05:20.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 100 reason = Timeout
105702026-03-24T21:05:20.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
105712026-03-24T21:05:20.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105722026-03-24T21:05:20.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105732026-03-24T21:05:20.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
105742026-03-24T21:05:20.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105752026-03-24T21:05:20.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105762026-03-24T21:05:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
105772026-03-24T21:05:20.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
105782026-03-24T21:05:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
105792026-03-24T21:05:20.539ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
105802026-03-24T21:05:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
105812026-03-24T21:05:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
105822026-03-24T21:05:20.541ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 536e3610-cdc3-4254-98ba-07880214c42c original_time_started = 2026-03-24 21:05:06.993628 UTC
105832026-03-24T21:05:20.541ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.868582ms iteration = 45 status = {"Loaded":{"collection_id":"536e3610-cdc3-4254-98ba-07880214c42c","time_loaded":"2026-03-24T21:05:20.533644459Z","time_started":"2026-03-24T21:05:06.993628Z"}}
105842026-03-24T21:05:20.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
105852026-03-24T21:05:20.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.051157ms iteration = 100 status = {"config_updated":false}
105862026-03-24T21:05:20.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 34 reason = Timeout
105872026-03-24T21:05:20.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 34 reason = Timeout
105882026-03-24T21:05:20.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
105892026-03-24T21:05:20.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
105902026-03-24T21:05:20.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
105912026-03-24T21:05:20.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
105922026-03-24T21:05:20.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
105932026-03-24T21:05:20.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
105942026-03-24T21:05:20.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105952026-03-24T21:05:20.579ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105962026-03-24T21:05:20.580ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
105972026-03-24T21:05:20.580ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
105982026-03-24T21:05:20.580ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
105992026-03-24T21:05:20.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
106002026-03-24T21:05:20.582ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.915399ms iteration = 34 status = "NoSitrep"
106012026-03-24T21:05:20.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
106022026-03-24T21:05:20.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
106032026-03-24T21:05:20.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
106042026-03-24T21:05:20.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
106052026-03-24T21:05:20.585ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106062026-03-24T21:05:20.586ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
106072026-03-24T21:05:20.589ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
106082026-03-24T21:05:20.590ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
106092026-03-24T21:05:20.590ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.770832ms iteration = 34 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
106102026-03-24T21:05:23.551ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 4899af95-ee0e-4743-aa3e-269bf201fb4c uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
106112026-03-24T21:05:23.551ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 4899af95-ee0e-4743-aa3e-269bf201fb4c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
106122026-03-24T21:05:23.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 4899af95-ee0e-4743-aa3e-269bf201fb4c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
106132026-03-24T21:05:23.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 3516 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 4899af95-ee0e-4743-aa3e-269bf201fb4c response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
106142026-03-24T21:05:23.555ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = cfaf9a3d-6458-4982-a1fd-60bd0f7bb76b uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
106152026-03-24T21:05:23.555ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = cfaf9a3d-6458-4982-a1fd-60bd0f7bb76b roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
106162026-03-24T21:05:23.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = cfaf9a3d-6458-4982-a1fd-60bd0f7bb76b resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
106172026-03-24T21:05:23.557ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2252 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = cfaf9a3d-6458-4982-a1fd-60bd0f7bb76b response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
106182026-03-24T21:05:25.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
106192026-03-24T21:05:25.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 101 reason = Timeout
106202026-03-24T21:05:25.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106212026-03-24T21:05:25.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
106222026-03-24T21:05:25.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
106232026-03-24T21:05:25.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
106242026-03-24T21:05:25.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106252026-03-24T21:05:25.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106262026-03-24T21:05:25.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
106272026-03-24T21:05:25.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106282026-03-24T21:05:25.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
106292026-03-24T21:05:25.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
106302026-03-24T21:05:25.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
106312026-03-24T21:05:25.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
106322026-03-24T21:05:25.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
106332026-03-24T21:05:25.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.950857ms iteration = 101 status = {"config_updated":false}
106342026-03-24T21:05:25.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106352026-03-24T21:05:25.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
106362026-03-24T21:05:25.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106372026-03-24T21:05:25.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
106382026-03-24T21:05:25.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106392026-03-24T21:05:25.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
106402026-03-24T21:05:25.558ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
106412026-03-24T21:05:25.558ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.744608ms iteration = 51 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
106422026-03-24T21:05:25.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
106432026-03-24T21:05:25.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
106442026-03-24T21:05:25.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
106452026-03-24T21:05:25.563ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
106462026-03-24T21:05:25.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
106472026-03-24T21:05:25.564ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.101454ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
106482026-03-24T21:05:28.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 432 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 40efde3d-11b7-4119-b461-6c52483b2dbb response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
106492026-03-24T21:05:30.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 102 reason = Timeout
106502026-03-24T21:05:30.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
106512026-03-24T21:05:30.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
106522026-03-24T21:05:30.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106532026-03-24T21:05:30.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
106542026-03-24T21:05:30.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
106552026-03-24T21:05:30.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
106562026-03-24T21:05:30.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
106572026-03-24T21:05:30.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.242616ms iteration = 102 status = {"config_updated":false}
106582026-03-24T21:05:35.521ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 18 reason = Timeout
106592026-03-24T21:05:35.521ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
106602026-03-24T21:05:35.521ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
106612026-03-24T21:05:35.521ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
106622026-03-24T21:05:35.521ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
106632026-03-24T21:05:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
106642026-03-24T21:05:35.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
106652026-03-24T21:05:35.524ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1321, tv_nsec: 737811243 } })
106662026-03-24T21:05:35.525ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1321, tv_nsec: 738611535 } })
106672026-03-24T21:05:35.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
106682026-03-24T21:05:35.526ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1321, tv_nsec: 739345815 } })
106692026-03-24T21:05:35.527ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.417586ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
106702026-03-24T21:05:35.527ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
106712026-03-24T21:05:35.527ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 5.863888ms iteration = 18 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
106722026-03-24T21:05:35.527ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
106732026-03-24T21:05:35.527ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
106742026-03-24T21:05:35.528ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9dcff63-f04e-442f-adb4-2ded0befa53b", "content-length": "26", "date": "Tue, 24 Mar 2026 21:05:35 GMT"} })
106752026-03-24T21:05:35.528ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
106762026-03-24T21:05:35.528ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
106772026-03-24T21:05:35.529ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1321, tv_nsec: 742288024 } })
106782026-03-24T21:05:35.531ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 10.096913ms iteration = 18 status = {}
106792026-03-24T21:05:35.533ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
106802026-03-24T21:05:35.533ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
106812026-03-24T21:05:35.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
106822026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106832026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
106842026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 46 reason = Timeout
106852026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
106862026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
106872026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 103 reason = Timeout
106882026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
106892026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
106902026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
106912026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
106922026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
106932026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
106942026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
106952026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
106962026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
106972026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106982026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
106992026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
107002026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
107012026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
107022026-03-24T21:05:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
107032026-03-24T21:05:35.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107042026-03-24T21:05:35.536ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107052026-03-24T21:05:35.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107062026-03-24T21:05:35.536ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107072026-03-24T21:05:35.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107082026-03-24T21:05:35.537ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107092026-03-24T21:05:35.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
107102026-03-24T21:05:35.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
107112026-03-24T21:05:35.540ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107122026-03-24T21:05:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
107132026-03-24T21:05:35.541ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107142026-03-24T21:05:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107152026-03-24T21:05:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
107162026-03-24T21:05:35.542ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
107172026-03-24T21:05:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
107182026-03-24T21:05:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
107192026-03-24T21:05:35.543ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
107202026-03-24T21:05:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
107212026-03-24T21:05:35.543ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
107222026-03-24T21:05:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
107232026-03-24T21:05:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
107242026-03-24T21:05:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
107252026-03-24T21:05:35.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107262026-03-24T21:05:35.544ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1321, tv_nsec: 757554376 } })
107272026-03-24T21:05:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
107282026-03-24T21:05:35.544ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
107292026-03-24T21:05:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
107302026-03-24T21:05:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
107312026-03-24T21:05:35.544ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
107322026-03-24T21:05:35.545ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1321, tv_nsec: 758485461 } })
107332026-03-24T21:05:35.546ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
107342026-03-24T21:05:35.546ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
107352026-03-24T21:05:35.546ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 12.304052ms iteration = 103 status = {"config_updated":false}
107362026-03-24T21:05:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
107372026-03-24T21:05:35.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
107382026-03-24T21:05:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
107392026-03-24T21:05:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
107402026-03-24T21:05:35.547ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
107412026-03-24T21:05:35.547ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
107422026-03-24T21:05:35.547ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
107432026-03-24T21:05:35.548ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
107442026-03-24T21:05:35.548ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3752d895-121e-4178-93b3-fc3716f892b9", "content-length": "26", "date": "Tue, 24 Mar 2026 21:05:35 GMT"} })
107452026-03-24T21:05:35.548ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
107462026-03-24T21:05:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
107472026-03-24T21:05:35.548ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
107482026-03-24T21:05:35.550ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1321, tv_nsec: 763487286 } })
107492026-03-24T21:05:35.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107502026-03-24T21:05:35.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
107512026-03-24T21:05:35.550ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 536e3610-cdc3-4254-98ba-07880214c42c original_time_started = 2026-03-24 21:05:06.993628 UTC
107522026-03-24T21:05:35.550ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 16.729421ms iteration = 46 status = {"Loaded":{"collection_id":"536e3610-cdc3-4254-98ba-07880214c42c","time_loaded":"2026-03-24T21:05:35.534041161Z","time_started":"2026-03-24T21:05:06.993628Z"}}
107532026-03-24T21:05:35.551ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107542026-03-24T21:05:35.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107552026-03-24T21:05:35.555ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 21.983113ms iteration = 18 status = 0
107562026-03-24T21:05:35.557ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1321, tv_nsec: 770565837 } })
107572026-03-24T21:05:35.558ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 18 reason = Timeout
107582026-03-24T21:05:35.558ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
107592026-03-24T21:05:35.558ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
107602026-03-24T21:05:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
107612026-03-24T21:05:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
107622026-03-24T21:05:35.559ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
107632026-03-24T21:05:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
107642026-03-24T21:05:35.559ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
107652026-03-24T21:05:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
107662026-03-24T21:05:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
107672026-03-24T21:05:35.559ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107682026-03-24T21:05:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107692026-03-24T21:05:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
107702026-03-24T21:05:35.560ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
107712026-03-24T21:05:35.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
107722026-03-24T21:05:35.560ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
107732026-03-24T21:05:35.560ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
107742026-03-24T21:05:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
107752026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
107762026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
107772026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
107782026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
107792026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
107802026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
107812026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
107822026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
107832026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
107842026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
107852026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
107862026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
107872026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
107882026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
107892026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
107902026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
107912026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
107922026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
107932026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
107942026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
107952026-03-24T21:05:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
107962026-03-24T21:05:35.564ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
107972026-03-24T21:05:35.564ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107982026-03-24T21:05:35.564ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.995331ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
107992026-03-24T21:05:35.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
108002026-03-24T21:05:35.565ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
108012026-03-24T21:05:35.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
108022026-03-24T21:05:35.566ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108032026-03-24T21:05:35.566ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 5.5501ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
108042026-03-24T21:05:35.566ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 18.472468ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
108052026-03-24T21:05:35.567ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 9.288121ms iteration = 18 status = {"errors":[],"requests_created_ok":[]}
108062026-03-24T21:05:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
108072026-03-24T21:05:35.569ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
108082026-03-24T21:05:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
108092026-03-24T21:05:35.576ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 18 reason = Timeout
108102026-03-24T21:05:35.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 35 reason = Timeout
108112026-03-24T21:05:35.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 35 reason = Timeout
108122026-03-24T21:05:35.576ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 24.381\u{b5}s iteration = 18 status = {"disabled":true,"errors":[],"sps":[]}
108132026-03-24T21:05:35.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
108142026-03-24T21:05:35.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
108152026-03-24T21:05:35.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
108162026-03-24T21:05:35.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
108172026-03-24T21:05:35.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
108182026-03-24T21:05:35.577ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
108192026-03-24T21:05:35.577ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 16.775632ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
108202026-03-24T21:05:35.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
108212026-03-24T21:05:35.579ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108222026-03-24T21:05:35.580ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108232026-03-24T21:05:35.583ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
108242026-03-24T21:05:35.584ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
108252026-03-24T21:05:35.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
108262026-03-24T21:05:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
108272026-03-24T21:05:35.585ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
108282026-03-24T21:05:35.586ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
108292026-03-24T21:05:35.586ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
108302026-03-24T21:05:35.586ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
108312026-03-24T21:05:35.586ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
108322026-03-24T21:05:35.586ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
108332026-03-24T21:05:35.586ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
108342026-03-24T21:05:35.586ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
108352026-03-24T21:05:35.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
108362026-03-24T21:05:35.587ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
108372026-03-24T21:05:35.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
108382026-03-24T21:05:35.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 11.742717ms iteration = 35 status = "NoSitrep"
108392026-03-24T21:05:35.588ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
108402026-03-24T21:05:35.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
108412026-03-24T21:05:35.590ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108422026-03-24T21:05:35.592ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
108432026-03-24T21:05:35.595ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
108442026-03-24T21:05:35.595ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
108452026-03-24T21:05:35.600ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
108462026-03-24T21:05:35.600ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 66.213866ms iteration = 52 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
108472026-03-24T21:05:35.604ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
108482026-03-24T21:05:35.604ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
108492026-03-24T21:05:35.604ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 27.705258ms iteration = 35 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
108502026-03-24T21:05:35.620ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
108512026-03-24T21:05:35.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
108522026-03-24T21:05:35.622ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108532026-03-24T21:05:35.623ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
108542026-03-24T21:05:35.623ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
108552026-03-24T21:05:35.623ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
108562026-03-24T21:05:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
108572026-03-24T21:05:35.624ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
108582026-03-24T21:05:35.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
108592026-03-24T21:05:35.625ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
108602026-03-24T21:05:35.625ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
108612026-03-24T21:05:35.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
108622026-03-24T21:05:35.626ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
108632026-03-24T21:05:35.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
108642026-03-24T21:05:35.630ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108652026-03-24T21:05:35.631ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 70.80455ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
108662026-03-24T21:05:35.633ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
108672026-03-24T21:05:35.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
108682026-03-24T21:05:35.634ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
108692026-03-24T21:05:35.634ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
108702026-03-24T21:05:35.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
108712026-03-24T21:05:35.635ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.847018ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
108722026-03-24T21:05:35.682ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
108732026-03-24T21:05:35.682ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 148.282318ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
108742026-03-24T21:05:35.683ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
108752026-03-24T21:05:35.683ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
108762026-03-24T21:05:35.683ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
108772026-03-24T21:05:35.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
108782026-03-24T21:05:35.691ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108792026-03-24T21:05:35.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 130.708924ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
108802026-03-24T21:05:35.693ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
108812026-03-24T21:05:35.693ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
108822026-03-24T21:05:35.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
108832026-03-24T21:05:35.696ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
108842026-03-24T21:05:35.696ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
108852026-03-24T21:05:35.702ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4300814-0522-4513-b589-9e5cdeecea79", "content-length": "372", "date": "Tue, 24 Mar 2026 21:05:35 GMT"} })
108862026-03-24T21:05:35.702ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
108872026-03-24T21:05:35.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
108882026-03-24T21:05:35.728ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
108892026-03-24T21:05:35.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
108902026-03-24T21:05:35.742ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
108912026-03-24T21:05:35.742ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
108922026-03-24T21:05:35.742ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 181.685009ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
108932026-03-24T21:05:35.748ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 214.878315ms iteration = 30 status = {"physical_disks_added":0}
108942026-03-24T21:05:35.763ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
108952026-03-24T21:05:35.763ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
108962026-03-24T21:05:35.768ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f951d328-93bf-426e-ac91-a343dce2932d", "date": "Tue, 24 Mar 2026 21:05:35 GMT"} })
108972026-03-24T21:05:35.768ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
108982026-03-24T21:05:35.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
108992026-03-24T21:05:35.770ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
109002026-03-24T21:05:35.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
109012026-03-24T21:05:35.774ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
109022026-03-24T21:05:35.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
109032026-03-24T21:05:35.777ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
109042026-03-24T21:05:35.777ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
109052026-03-24T21:05:35.784ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "704eef56-7307-405d-9652-f57ef0c1fa80", "date": "Tue, 24 Mar 2026 21:05:35 GMT"} })
109062026-03-24T21:05:35.785ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 224.103393ms iteration = 27 status = {}
109072026-03-24T21:05:35.849ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "3e838570-01f0-4df4-b795-dc74e6072bcf", "date": "Tue, 24 Mar 2026 21:05:35 GMT"} })
109082026-03-24T21:05:35.849ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
109092026-03-24T21:05:35.849ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109102026-03-24T21:05:35.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109112026-03-24T21:05:35.878ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
109122026-03-24T21:05:35.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
109132026-03-24T21:05:35.880ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109142026-03-24T21:05:35.882ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
109152026-03-24T21:05:35.882ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
109162026-03-24T21:05:35.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
109172026-03-24T21:05:35.884ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109182026-03-24T21:05:35.886ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
109192026-03-24T21:05:35.886ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
109202026-03-24T21:05:35.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
109212026-03-24T21:05:35.889ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
109222026-03-24T21:05:35.890ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ce06ed6-f875-42a4-9f19-c0d862c300ab", "content-length": "691", "date": "Tue, 24 Mar 2026 21:05:35 GMT"} })
109232026-03-24T21:05:35.890ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
109242026-03-24T21:05:35.890ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
109252026-03-24T21:05:35.890ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
109262026-03-24T21:05:35.890ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 356.00337ms iteration = 18 status = {}
109272026-03-24T21:05:35.899ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109282026-03-24T21:05:35.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109292026-03-24T21:05:35.901ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109302026-03-24T21:05:35.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109312026-03-24T21:05:35.904ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109322026-03-24T21:05:35.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109332026-03-24T21:05:35.905ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 372.038842ms iteration = 18 status = {}
109342026-03-24T21:05:36.281ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 3
109352026-03-24T21:05:36.281ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221
109362026-03-24T21:05:36.281ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
109372026-03-24T21:05:36.282ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:57321
109382026-03-24T21:05:36.282ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:57321 req_id = 1561d5bb-97ca-4f41-9c0a-ee2accb3080a uri = /metrics/producers version = 1.0.0
109392026-03-24T21:05:36.282ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:57321 req_id = 1561d5bb-97ca-4f41-9c0a-ee2accb3080a roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
109402026-03-24T21:05:36.282ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:57321 req_id = 1561d5bb-97ca-4f41-9c0a-ee2accb3080a resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
109412026-03-24T21:05:36.341ZINFOnexus: assigned collector to new producer collector_id = a309df7c-2a20-4201-8e80-eb9f7e355177 file = nexus/src/app/oximeter.rs:76 producer_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb
109422026-03-24T21:05:36.341ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 59430 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:57321 req_id = 1561d5bb-97ca-4f41-9c0a-ee2accb3080a response_code = 201 uri = /metrics/producers
109432026-03-24T21:05:36.341ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "1561d5bb-97ca-4f41-9c0a-ee2accb3080a", "content-length": "41", "date": "Tue, 24 Mar 2026 21:05:36 GMT"} })
109442026-03-24T21:05:36.341ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
109452026-03-24T21:05:38.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 370 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = e2be46ef-1717-4da9-977c-79d6703bb0a0 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
109462026-03-24T21:05:40.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 104 reason = Timeout
109472026-03-24T21:05:40.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
109482026-03-24T21:05:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
109492026-03-24T21:05:40.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109502026-03-24T21:05:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
109512026-03-24T21:05:40.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
109522026-03-24T21:05:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
109532026-03-24T21:05:40.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
109542026-03-24T21:05:40.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.095981ms iteration = 104 status = {"config_updated":false}
109552026-03-24T21:05:45.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
109562026-03-24T21:05:45.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 105 reason = Timeout
109572026-03-24T21:05:45.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
109582026-03-24T21:05:45.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
109592026-03-24T21:05:45.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
109602026-03-24T21:05:45.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
109612026-03-24T21:05:45.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109622026-03-24T21:05:45.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109632026-03-24T21:05:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
109642026-03-24T21:05:45.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
109652026-03-24T21:05:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
109662026-03-24T21:05:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
109672026-03-24T21:05:45.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
109682026-03-24T21:05:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
109692026-03-24T21:05:45.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
109702026-03-24T21:05:45.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
109712026-03-24T21:05:45.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
109722026-03-24T21:05:45.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.15994ms iteration = 105 status = {"config_updated":false}
109732026-03-24T21:05:45.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109742026-03-24T21:05:45.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
109752026-03-24T21:05:45.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
109762026-03-24T21:05:45.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
109772026-03-24T21:05:45.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
109782026-03-24T21:05:45.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
109792026-03-24T21:05:45.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
109802026-03-24T21:05:45.562ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
109812026-03-24T21:05:45.562ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.754545ms iteration = 53 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
109822026-03-24T21:05:45.609ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
109832026-03-24T21:05:45.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
109842026-03-24T21:05:45.659ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 97.817008ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
109852026-03-24T21:05:48.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 648 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 623b79b3-3c3a-4433-9124-c8516b35af90 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
109862026-03-24T21:05:50.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 47 reason = Timeout
109872026-03-24T21:05:50.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 106 reason = Timeout
109882026-03-24T21:05:50.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
109892026-03-24T21:05:50.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
109902026-03-24T21:05:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
109912026-03-24T21:05:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
109922026-03-24T21:05:50.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109932026-03-24T21:05:50.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109942026-03-24T21:05:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
109952026-03-24T21:05:50.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
109962026-03-24T21:05:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
109972026-03-24T21:05:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
109982026-03-24T21:05:50.539ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
109992026-03-24T21:05:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
110002026-03-24T21:05:50.541ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 536e3610-cdc3-4254-98ba-07880214c42c original_time_started = 2026-03-24 21:05:06.993628 UTC
110012026-03-24T21:05:50.541ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.403577ms iteration = 47 status = {"Loaded":{"collection_id":"536e3610-cdc3-4254-98ba-07880214c42c","time_loaded":"2026-03-24T21:05:50.532940305Z","time_started":"2026-03-24T21:05:06.993628Z"}}
110022026-03-24T21:05:50.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
110032026-03-24T21:05:50.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.472898ms iteration = 106 status = {"config_updated":false}
110042026-03-24T21:05:50.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 36 reason = Timeout
110052026-03-24T21:05:50.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
110062026-03-24T21:05:50.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 36 reason = Timeout
110072026-03-24T21:05:50.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
110082026-03-24T21:05:50.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
110092026-03-24T21:05:50.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
110102026-03-24T21:05:50.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
110112026-03-24T21:05:50.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
110122026-03-24T21:05:50.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110132026-03-24T21:05:50.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110142026-03-24T21:05:50.580ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
110152026-03-24T21:05:50.580ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
110162026-03-24T21:05:50.580ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
110172026-03-24T21:05:50.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
110182026-03-24T21:05:50.582ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.712132ms iteration = 36 status = "NoSitrep"
110192026-03-24T21:05:50.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
110202026-03-24T21:05:50.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
110212026-03-24T21:05:50.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
110222026-03-24T21:05:50.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
110232026-03-24T21:05:50.585ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110242026-03-24T21:05:50.586ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
110252026-03-24T21:05:50.590ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
110262026-03-24T21:05:50.590ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
110272026-03-24T21:05:50.590ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 14.409119ms iteration = 36 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
110282026-03-24T21:05:53.552ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = afa49bb9-7ba7-48f5-92cd-c6ae086640cc uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
110292026-03-24T21:05:53.552ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = afa49bb9-7ba7-48f5-92cd-c6ae086640cc roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
110302026-03-24T21:05:53.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = afa49bb9-7ba7-48f5-92cd-c6ae086640cc resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
110312026-03-24T21:05:53.554ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2591 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = afa49bb9-7ba7-48f5-92cd-c6ae086640cc response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
110322026-03-24T21:05:53.554ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 294183c2-314e-4014-94ba-acae4218b3ba uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
110332026-03-24T21:05:53.555ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 294183c2-314e-4014-94ba-acae4218b3ba roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
110342026-03-24T21:05:53.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 294183c2-314e-4014-94ba-acae4218b3ba resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
110352026-03-24T21:05:53.557ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2472 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 294183c2-314e-4014-94ba-acae4218b3ba response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
110362026-03-24T21:05:55.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
110372026-03-24T21:05:55.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 107 reason = Timeout
110382026-03-24T21:05:55.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110392026-03-24T21:05:55.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
110402026-03-24T21:05:55.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
110412026-03-24T21:05:55.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
110422026-03-24T21:05:55.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110432026-03-24T21:05:55.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110442026-03-24T21:05:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
110452026-03-24T21:05:55.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110462026-03-24T21:05:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
110472026-03-24T21:05:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
110482026-03-24T21:05:55.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
110492026-03-24T21:05:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
110502026-03-24T21:05:55.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110512026-03-24T21:05:55.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
110522026-03-24T21:05:55.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
110532026-03-24T21:05:55.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.191204ms iteration = 107 status = {"config_updated":false}
110542026-03-24T21:05:55.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110552026-03-24T21:05:55.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
110562026-03-24T21:05:55.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
110572026-03-24T21:05:55.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
110582026-03-24T21:05:55.557ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
110592026-03-24T21:05:55.557ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.196422ms iteration = 54 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
110602026-03-24T21:05:55.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
110612026-03-24T21:05:55.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
110622026-03-24T21:05:55.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
110632026-03-24T21:05:55.562ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
110642026-03-24T21:05:55.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
110652026-03-24T21:05:55.564ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.823787ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
110662026-03-24T21:05:58.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 373 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = dc25fd25-5a60-47ef-8e34-5399f746f4ea response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
110672026-03-24T21:06:00.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 108 reason = Timeout
110682026-03-24T21:06:00.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
110692026-03-24T21:06:00.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
110702026-03-24T21:06:00.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110712026-03-24T21:06:00.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
110722026-03-24T21:06:00.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
110732026-03-24T21:06:00.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
110742026-03-24T21:06:00.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
110752026-03-24T21:06:00.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.491012ms iteration = 108 status = {"config_updated":false}
110762026-03-24T21:06:05.516ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
110772026-03-24T21:06:05.516ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
110782026-03-24T21:06:05.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
110792026-03-24T21:06:05.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
110802026-03-24T21:06:05.517ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
110812026-03-24T21:06:05.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
110822026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110832026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
110842026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
110852026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
110862026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
110872026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
110882026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
110892026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
110902026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
110912026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
110922026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
110932026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 19 reason = Timeout
110942026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
110952026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
110962026-03-24T21:06:05.520ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2026-03-24 20:56:05.520755963 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
110972026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
110982026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
110992026-03-24T21:06:05.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
111002026-03-24T21:06:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
111012026-03-24T21:06:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
111022026-03-24T21:06:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
111032026-03-24T21:06:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
111042026-03-24T21:06:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
111052026-03-24T21:06:05.521ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
111062026-03-24T21:06:05.521ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
111072026-03-24T21:06:05.521ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
111082026-03-24T21:06:05.521ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = ExternalNetworking step id = Ensure
111092026-03-24T21:06:05.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
111102026-03-24T21:06:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("93b6e989-1f7d-4884-aec9-ecbc076ed6c4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 735348418 } })
111112026-03-24T21:06:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("27f3a9c5-67c7-41c5-994d-9db513cda557.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2f7ea2ed-fd45-4cd7-9fb1-f48119541118.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 735348378 } })
111122026-03-24T21:06:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
111132026-03-24T21:06:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
111142026-03-24T21:06:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
111152026-03-24T21:06:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.86611ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
111162026-03-24T21:06:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
111172026-03-24T21:06:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.815089ms iteration = 10 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
111182026-03-24T21:06:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
111192026-03-24T21:06:05.522ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
111202026-03-24T21:06:05.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
111212026-03-24T21:06:05.523ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111222026-03-24T21:06:05.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
111232026-03-24T21:06:05.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
111242026-03-24T21:06:05.524ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 738472622 } })
111252026-03-24T21:06:05.524ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111262026-03-24T21:06:05.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "7e03e6f6-11de-4ac1-b017-438d6f678aae", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
111272026-03-24T21:06:05.525ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
111282026-03-24T21:06:05.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "37452454-03a7-4fda-b2ae-707cf0cbba70", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
111292026-03-24T21:06:05.525ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
111302026-03-24T21:06:05.525ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
111312026-03-24T21:06:05.525ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 4.729377ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
111322026-03-24T21:06:05.525ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
111332026-03-24T21:06:05.525ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 739290164 } })
111342026-03-24T21:06:05.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
111352026-03-24T21:06:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "44138383-2105-4930-a9d4-a4d35f6b0b24", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111362026-03-24T21:06:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111372026-03-24T21:06:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "52d5b456-61a4-47b0-8bfb-66aada0f6459", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111382026-03-24T21:06:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111392026-03-24T21:06:05.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
111402026-03-24T21:06:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a2d69b80-9b55-45b2-b2b8-42d370601876", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111412026-03-24T21:06:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111422026-03-24T21:06:05.526ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
111432026-03-24T21:06:05.526ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 5.619861ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
111442026-03-24T21:06:05.526ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 740228840 } })
111452026-03-24T21:06:05.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
111462026-03-24T21:06:05.527ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
111472026-03-24T21:06:05.527ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 6.413663ms iteration = 19 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
111482026-03-24T21:06:05.527ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2026-03-24 20:56:05.520755963 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
111492026-03-24T21:06:05.527ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.944807ms iteration = 10 status = {"expiration":"2026-03-24T20:56:05.520755963Z","pruned":[]}
111502026-03-24T21:06:05.527ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
111512026-03-24T21:06:05.527ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
111522026-03-24T21:06:05.527ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
111532026-03-24T21:06:05.527ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
111542026-03-24T21:06:05.528ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.360429ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
111552026-03-24T21:06:05.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
111562026-03-24T21:06:05.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
111572026-03-24T21:06:05.528ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d5f6078-9f04-45b1-b7b0-aa4db5240251", "content-length": "26", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} })
111582026-03-24T21:06:05.528ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
111592026-03-24T21:06:05.528ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
111602026-03-24T21:06:05.529ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 743222110 } })
111612026-03-24T21:06:05.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
111622026-03-24T21:06:05.532ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 11.659295ms iteration = 19 status = {}
111632026-03-24T21:06:05.532ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
111642026-03-24T21:06:05.532ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
111652026-03-24T21:06:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
111662026-03-24T21:06:05.533ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
111672026-03-24T21:06:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
111682026-03-24T21:06:05.533ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
111692026-03-24T21:06:05.533ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
111702026-03-24T21:06:05.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
111712026-03-24T21:06:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
111722026-03-24T21:06:05.533ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
111732026-03-24T21:06:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
111742026-03-24T21:06:05.533ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
111752026-03-24T21:06:05.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
111762026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
111772026-03-24T21:06:05.534ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
111782026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 24 reason = Timeout
111792026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
111802026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 28 reason = Timeout
111812026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
111822026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
111832026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111842026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Timeout
111852026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
111862026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
111872026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
111882026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
111892026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 48 reason = Timeout
111902026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
111912026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 109 reason = Timeout
111922026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
111932026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
111942026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
111952026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
111962026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
111972026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
111982026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
111992026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
112002026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
112012026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112022026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
112032026-03-24T21:06:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
112042026-03-24T21:06:05.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
112052026-03-24T21:06:05.539ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112062026-03-24T21:06:05.540ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
112072026-03-24T21:06:05.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
112082026-03-24T21:06:05.540ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
112092026-03-24T21:06:05.540ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
112102026-03-24T21:06:05.540ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
112112026-03-24T21:06:05.540ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
112122026-03-24T21:06:05.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
112132026-03-24T21:06:05.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
112142026-03-24T21:06:05.541ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
112152026-03-24T21:06:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
112162026-03-24T21:06:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
112172026-03-24T21:06:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
112182026-03-24T21:06:05.542ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
112192026-03-24T21:06:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
112202026-03-24T21:06:05.543ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112212026-03-24T21:06:05.544ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112222026-03-24T21:06:05.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112232026-03-24T21:06:05.544ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112242026-03-24T21:06:05.546ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112252026-03-24T21:06:05.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
112262026-03-24T21:06:05.546ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
112272026-03-24T21:06:05.546ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
112282026-03-24T21:06:05.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
112292026-03-24T21:06:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
112302026-03-24T21:06:05.547ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112312026-03-24T21:06:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
112322026-03-24T21:06:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
112332026-03-24T21:06:05.551ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
112342026-03-24T21:06:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
112352026-03-24T21:06:05.552ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
112362026-03-24T21:06:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
112372026-03-24T21:06:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
112382026-03-24T21:06:05.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
112392026-03-24T21:06:05.552ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112402026-03-24T21:06:05.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
112412026-03-24T21:06:05.553ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
112422026-03-24T21:06:05.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
112432026-03-24T21:06:05.553ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
112442026-03-24T21:06:05.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
112452026-03-24T21:06:05.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
112462026-03-24T21:06:05.554ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
112472026-03-24T21:06:05.554ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
112482026-03-24T21:06:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
112492026-03-24T21:06:05.555ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
112502026-03-24T21:06:05.555ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
112512026-03-24T21:06:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
112522026-03-24T21:06:05.558ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
112532026-03-24T21:06:05.558ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 22 reason = Timeout
112542026-03-24T21:06:05.558ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
112552026-03-24T21:06:05.558ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 15.52\u{b5}s iteration = 22 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
112562026-03-24T21:06:05.558ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 10 reason = Timeout
112572026-03-24T21:06:05.558ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
112582026-03-24T21:06:05.558ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
112592026-03-24T21:06:05.558ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 10 reason = Timeout
112602026-03-24T21:06:05.558ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
112612026-03-24T21:06:05.558ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
112622026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
112632026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
112642026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
112652026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
112662026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
112672026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
112682026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
112692026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
112702026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
112712026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
112722026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
112732026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
112742026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
112752026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
112762026-03-24T21:06:05.559ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
112772026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
112782026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
112792026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
112802026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.387801ms iteration = 19 status = 0
112812026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
112822026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112832026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
112842026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
112852026-03-24T21:06:05.560ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
112862026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
112872026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
112882026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
112892026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
112902026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
112912026-03-24T21:06:05.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
112922026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
112932026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
112942026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
112952026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
112962026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
112972026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
112982026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
112992026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113002026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
113012026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113022026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 7.119632ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
113032026-03-24T21:06:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
113042026-03-24T21:06:05.562ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
113052026-03-24T21:06:05.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
113062026-03-24T21:06:05.564ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
113072026-03-24T21:06:05.564ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
113082026-03-24T21:06:05.564ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
113092026-03-24T21:06:05.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
113102026-03-24T21:06:05.568ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
113112026-03-24T21:06:05.568ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
113122026-03-24T21:06:05.568ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 12.753433ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
113132026-03-24T21:06:05.568ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
113142026-03-24T21:06:05.568ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 9.63724ms iteration = 10 status = {"by_rx":{},"error":null}
113152026-03-24T21:06:05.569ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113162026-03-24T21:06:05.571ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113172026-03-24T21:06:05.573ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
113182026-03-24T21:06:05.574ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
113192026-03-24T21:06:05.574ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
113202026-03-24T21:06:05.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
113212026-03-24T21:06:05.575ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113222026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113232026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): activating background_task = trust_quorum_manager iteration = 10 reason = Timeout
113242026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): activating background_task = attached_subnet_manager iteration = 10 reason = Timeout
113252026-03-24T21:06:05.576ZINFOnexus (BackgroundTasks): starting attached subnet manager work background_task = attached_subnet_manager file = nexus/src/app/background/tasks/attached_subnets.rs:374
113262026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
113272026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 10 reason = Timeout
113282026-03-24T21:06:05.576ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
113292026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
113302026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 19 reason = Timeout
113312026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 37 reason = Timeout
113322026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 37 reason = Timeout
113332026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.74\u{b5}s iteration = 19 status = {"disabled":true,"errors":[],"sps":[]}
113342026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
113352026-03-24T21:06:05.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
113362026-03-24T21:06:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
113372026-03-24T21:06:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
113382026-03-24T21:06:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
113392026-03-24T21:06:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
113402026-03-24T21:06:05.577ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 791047380 } })
113412026-03-24T21:06:05.578ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
113422026-03-24T21:06:05.580ZDEBGnexus (BackgroundTasks): roles background_task = attached_subnet_manager roles = RoleSet { roles: {} }
113432026-03-24T21:06:05.582ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
113442026-03-24T21:06:05.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = attached_subnet_manager resource = Database result = Ok(())
113452026-03-24T21:06:05.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
113462026-03-24T21:06:05.583ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
113472026-03-24T21:06:05.583ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
113482026-03-24T21:06:05.583ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
113492026-03-24T21:06:05.583ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
113502026-03-24T21:06:05.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
113512026-03-24T21:06:05.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
113522026-03-24T21:06:05.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
113532026-03-24T21:06:05.584ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
113542026-03-24T21:06:05.584ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
113552026-03-24T21:06:05.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
113562026-03-24T21:06:05.584ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113572026-03-24T21:06:05.585ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 38.91872ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
113582026-03-24T21:06:05.587ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
113592026-03-24T21:06:05.587ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 54.945562ms iteration = 5 status = {"nsuccess":5}
113602026-03-24T21:06:05.587ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
113612026-03-24T21:06:05.587ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113622026-03-24T21:06:05.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113632026-03-24T21:06:05.590ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
113642026-03-24T21:06:05.599ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
113652026-03-24T21:06:05.599ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
113662026-03-24T21:06:05.599ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
113672026-03-24T21:06:05.599ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2026-03-24 20:57:05.047625 UTC dns_group = internal generation = 1
113682026-03-24T21:06:05.599ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 83.209214ms iteration = 10 status = {"generation":1}
113692026-03-24T21:06:05.599ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113702026-03-24T21:06:05.602ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
113712026-03-24T21:06:05.605ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113722026-03-24T21:06:05.605ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
113732026-03-24T21:06:05.605ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113742026-03-24T21:06:05.606ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
113752026-03-24T21:06:05.607ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
113762026-03-24T21:06:05.607ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
113772026-03-24T21:06:05.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
113782026-03-24T21:06:05.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
113792026-03-24T21:06:05.608ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
113802026-03-24T21:06:05.609ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
113812026-03-24T21:06:05.609ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
113822026-03-24T21:06:05.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
113832026-03-24T21:06:05.610ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
113842026-03-24T21:06:05.611ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = SledList step id = Fetch
113852026-03-24T21:06:05.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
113862026-03-24T21:06:05.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
113872026-03-24T21:06:05.611ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Fleet result = Ok(())
113882026-03-24T21:06:05.611ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
113892026-03-24T21:06:05.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
113902026-03-24T21:06:05.614ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 55.193498ms iteration = 10 status = {"dispatched":[],"errors":[],"glob_version":"242.0.0","globs_reprocessed":{},"no_receivers":[]}
113912026-03-24T21:06:05.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113922026-03-24T21:06:05.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113932026-03-24T21:06:05.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
113942026-03-24T21:06:05.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
113952026-03-24T21:06:05.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
113962026-03-24T21:06:05.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
113972026-03-24T21:06:05.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113982026-03-24T21:06:05.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
113992026-03-24T21:06:05.657ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
114002026-03-24T21:06:05.657ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
114012026-03-24T21:06:05.657ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
114022026-03-24T21:06:05.657ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
114032026-03-24T21:06:05.657ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114042026-03-24T21:06:05.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
114052026-03-24T21:06:05.660ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114062026-03-24T21:06:05.660ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114072026-03-24T21:06:05.660ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114082026-03-24T21:06:05.662ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114092026-03-24T21:06:05.663ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
114102026-03-24T21:06:05.663ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114112026-03-24T21:06:05.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114122026-03-24T21:06:05.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114132026-03-24T21:06:05.663ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114142026-03-24T21:06:05.663ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114152026-03-24T21:06:05.664ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 87.896251ms iteration = 37 status = "NoSitrep"
114162026-03-24T21:06:05.664ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
114172026-03-24T21:06:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
114182026-03-24T21:06:05.665ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
114192026-03-24T21:06:05.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
114202026-03-24T21:06:05.666ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114212026-03-24T21:06:05.666ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
114222026-03-24T21:06:05.666ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
114232026-03-24T21:06:05.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114242026-03-24T21:06:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114252026-03-24T21:06:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
114262026-03-24T21:06:05.667ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114272026-03-24T21:06:05.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
114282026-03-24T21:06:05.669ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114292026-03-24T21:06:05.669ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 882984389 } })
114302026-03-24T21:06:05.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114312026-03-24T21:06:05.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
114322026-03-24T21:06:05.669ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
114332026-03-24T21:06:05.669ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
114342026-03-24T21:06:05.669ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
114352026-03-24T21:06:05.669ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
114362026-03-24T21:06:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
114372026-03-24T21:06:05.670ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 884020677 } })
114382026-03-24T21:06:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
114392026-03-24T21:06:05.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114402026-03-24T21:06:05.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
114412026-03-24T21:06:05.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114422026-03-24T21:06:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114432026-03-24T21:06:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
114442026-03-24T21:06:05.671ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
114452026-03-24T21:06:05.671ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
114462026-03-24T21:06:05.671ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
114472026-03-24T21:06:05.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
114482026-03-24T21:06:05.672ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "550091e2-e913-4074-aa03-4f34e0f78e37", "content-length": "26", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} })
114492026-03-24T21:06:05.672ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
114502026-03-24T21:06:05.672ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
114512026-03-24T21:06:05.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
114522026-03-24T21:06:05.674ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
114532026-03-24T21:06:05.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
114542026-03-24T21:06:05.674ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 888229320 } })
114552026-03-24T21:06:05.675ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114562026-03-24T21:06:05.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114572026-03-24T21:06:05.675ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
114582026-03-24T21:06:05.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114592026-03-24T21:06:05.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
114602026-03-24T21:06:05.678ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1351, tv_nsec: 892498506 } })
114612026-03-24T21:06:05.678ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
114622026-03-24T21:06:05.679ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114632026-03-24T21:06:05.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
114642026-03-24T21:06:05.679ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
114652026-03-24T21:06:05.679ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114662026-03-24T21:06:05.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114672026-03-24T21:06:05.680ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114682026-03-24T21:06:05.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114692026-03-24T21:06:05.684ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
114702026-03-24T21:06:05.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
114712026-03-24T21:06:05.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
114722026-03-24T21:06:05.684ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
114732026-03-24T21:06:05.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
114742026-03-24T21:06:05.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
114752026-03-24T21:06:05.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114762026-03-24T21:06:05.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114772026-03-24T21:06:05.686ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
114782026-03-24T21:06:05.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
114792026-03-24T21:06:05.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114802026-03-24T21:06:05.688ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 536e3610-cdc3-4254-98ba-07880214c42c original_time_started = 2026-03-24 21:05:06.993628 UTC
114812026-03-24T21:06:05.688ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 154.595118ms iteration = 48 status = {"Loaded":{"collection_id":"536e3610-cdc3-4254-98ba-07880214c42c","time_loaded":"2026-03-24T21:06:05.534087893Z","time_started":"2026-03-24T21:05:06.993628Z"}}
114822026-03-24T21:06:05.689ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
114832026-03-24T21:06:05.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
114842026-03-24T21:06:05.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 130.970452ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
114852026-03-24T21:06:05.692ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 133.634824ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
114862026-03-24T21:06:05.692ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
114872026-03-24T21:06:05.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
114882026-03-24T21:06:05.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
114892026-03-24T21:06:05.695ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
114902026-03-24T21:06:05.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
114912026-03-24T21:06:05.699ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114922026-03-24T21:06:05.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114932026-03-24T21:06:05.705ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
114942026-03-24T21:06:05.705ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114952026-03-24T21:06:05.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114962026-03-24T21:06:05.706ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
114972026-03-24T21:06:05.706ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 172.302537ms iteration = 109 status = {"config_updated":false}
114982026-03-24T21:06:05.707ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 146.272154ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
114992026-03-24T21:06:05.708ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115002026-03-24T21:06:05.709ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115012026-03-24T21:06:05.709ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = DeployNexusRecords step id = Ensure
115022026-03-24T21:06:05.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115032026-03-24T21:06:05.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115042026-03-24T21:06:05.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115052026-03-24T21:06:05.711ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 150.726454ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
115062026-03-24T21:06:05.712ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
115072026-03-24T21:06:05.712ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115082026-03-24T21:06:05.716ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
115092026-03-24T21:06:05.716ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
115102026-03-24T21:06:05.716ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
115112026-03-24T21:06:05.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
115122026-03-24T21:06:05.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115132026-03-24T21:06:05.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
115142026-03-24T21:06:05.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115152026-03-24T21:06:05.720ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
115162026-03-24T21:06:05.720ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
115172026-03-24T21:06:05.720ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
115182026-03-24T21:06:05.720ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115192026-03-24T21:06:05.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115202026-03-24T21:06:05.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
115212026-03-24T21:06:05.722ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
115222026-03-24T21:06:05.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
115232026-03-24T21:06:05.724ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
115242026-03-24T21:06:05.724ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
115252026-03-24T21:06:05.724ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 148.452603ms iteration = 37 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
115262026-03-24T21:06:05.729ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
115272026-03-24T21:06:05.729ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 195.638816ms iteration = 55 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
115282026-03-24T21:06:05.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115292026-03-24T21:06:05.739ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
115302026-03-24T21:06:05.739ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
115312026-03-24T21:06:05.741ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
115322026-03-24T21:06:05.742ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
115332026-03-24T21:06:05.742ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
115342026-03-24T21:06:05.742ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
115352026-03-24T21:06:05.742ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
115362026-03-24T21:06:05.742ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115372026-03-24T21:06:05.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
115382026-03-24T21:06:05.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115392026-03-24T21:06:05.749ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 188.573565ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
115402026-03-24T21:06:05.749ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
115412026-03-24T21:06:05.749ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
115422026-03-24T21:06:05.760ZINFOnexus (BackgroundTasks): Loaded 0 active trust quorum configurations from database background_task = trust_quorum_manager bgtask = TrustQuorumManager file = nexus/src/app/background/tasks/trust_quorum.rs:83
115432026-03-24T21:06:05.760ZDEBGnexus (BackgroundTasks): activation complete background_task = trust_quorum_manager elapsed = 184.552416ms iteration = 10 status = {"PerRackStatus":{"errors":[],"statuses":[]}}
115442026-03-24T21:06:05.764ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
115452026-03-24T21:06:05.764ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
115462026-03-24T21:06:05.764ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 230.958958ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
115472026-03-24T21:06:05.765ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
115482026-03-24T21:06:05.765ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
115492026-03-24T21:06:05.766ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
115502026-03-24T21:06:05.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
115512026-03-24T21:06:05.771ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
115522026-03-24T21:06:05.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
115532026-03-24T21:06:05.774ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115542026-03-24T21:06:05.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
115552026-03-24T21:06:05.779ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
115562026-03-24T21:06:05.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
115572026-03-24T21:06:05.781ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
115582026-03-24T21:06:05.781ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 247.042652ms iteration = 28 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
115592026-03-24T21:06:05.783ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
115602026-03-24T21:06:05.783ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
115612026-03-24T21:06:05.783ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
115622026-03-24T21:06:05.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
115632026-03-24T21:06:05.786ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115642026-03-24T21:06:05.791ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
115652026-03-24T21:06:05.791ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
115662026-03-24T21:06:05.791ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 231.873262ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
115672026-03-24T21:06:05.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
115682026-03-24T21:06:05.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
115692026-03-24T21:06:05.793ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
115702026-03-24T21:06:05.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
115712026-03-24T21:06:05.796ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
115722026-03-24T21:06:05.796ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
115732026-03-24T21:06:05.796ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2026-03-24 20:57:07.125641 UTC dns_group = external generation = 2
115742026-03-24T21:06:05.796ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 275.648683ms iteration = 10 status = {"generation":2}
115752026-03-24T21:06:05.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
115762026-03-24T21:06:05.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
115772026-03-24T21:06:05.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
115782026-03-24T21:06:05.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
115792026-03-24T21:06:05.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115802026-03-24T21:06:05.810ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
115812026-03-24T21:06:05.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
115822026-03-24T21:06:05.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
115832026-03-24T21:06:05.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
115842026-03-24T21:06:05.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
115852026-03-24T21:06:05.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115862026-03-24T21:06:05.823ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
115872026-03-24T21:06:05.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
115882026-03-24T21:06:05.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
115892026-03-24T21:06:05.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115902026-03-24T21:06:05.826ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 292.470526ms iteration = 31 status = {"physical_disks_added":0}
115912026-03-24T21:06:05.831ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
115922026-03-24T21:06:05.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
115932026-03-24T21:06:05.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
115942026-03-24T21:06:05.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
115952026-03-24T21:06:05.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
115962026-03-24T21:06:05.837ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
115972026-03-24T21:06:05.837ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
115982026-03-24T21:06:05.837ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 276.340671ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
115992026-03-24T21:06:05.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116002026-03-24T21:06:05.839ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2026-03-24T20:56:59.957750Z, time_modified: 2026-03-24T20:56:59.957750Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }) }
116012026-03-24T21:06:05.839ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 318.531909ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]}},"default_endpoint":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
116022026-03-24T21:06:05.840ZDEBGnexus (BackgroundTasks): listed attached subnets background_task = attached_subnet_manager n_subnets = 0
116032026-03-24T21:06:05.841ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
116042026-03-24T21:06:05.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
116052026-03-24T21:06:05.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
116062026-03-24T21:06:05.844ZDEBGnexus (BackgroundTasks): activation complete background_task = attached_subnet_manager elapsed = 267.902074ms iteration = 10 status = {"db_error":null,"dendrite":{"switch0":{"errors":[],"n_subnets_added":0,"n_subnets_removed":0,"n_total_subnets":0}},"sled":{}}
116072026-03-24T21:06:05.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
116082026-03-24T21:06:05.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
116092026-03-24T21:06:05.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116102026-03-24T21:06:05.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
116112026-03-24T21:06:05.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
116122026-03-24T21:06:05.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
116132026-03-24T21:06:05.853ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
116142026-03-24T21:06:05.853ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
116152026-03-24T21:06:05.855ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1edbb01e-fbe6-406f-90a7-6705fa2928fa", "content-length": "372", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} })
116162026-03-24T21:06:05.855ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116172026-03-24T21:06:05.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116182026-03-24T21:06:05.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
116192026-03-24T21:06:05.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
116202026-03-24T21:06:05.857ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = SledAgent step id = Ensure
116212026-03-24T21:06:05.857ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
116222026-03-24T21:06:05.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
116232026-03-24T21:06:05.859ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
116242026-03-24T21:06:05.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116252026-03-24T21:06:05.859ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
116262026-03-24T21:06:05.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
116272026-03-24T21:06:05.861ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "0927f593-b7db-4cdf-8b4f-ef16123bda6c", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} }) sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
116282026-03-24T21:06:05.862ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = FirewallRules step id = Ensure
116292026-03-24T21:06:05.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116302026-03-24T21:06:05.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116312026-03-24T21:06:05.862ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116322026-03-24T21:06:05.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116332026-03-24T21:06:05.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116342026-03-24T21:06:05.865ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
116352026-03-24T21:06:05.865ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
116362026-03-24T21:06:05.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
116372026-03-24T21:06:05.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
116382026-03-24T21:06:05.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
116392026-03-24T21:06:05.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
116402026-03-24T21:06:05.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116412026-03-24T21:06:05.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116422026-03-24T21:06:05.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116432026-03-24T21:06:05.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116442026-03-24T21:06:05.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116452026-03-24T21:06:05.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116462026-03-24T21:06:05.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 311.429678ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
116472026-03-24T21:06:05.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116482026-03-24T21:06:05.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116492026-03-24T21:06:05.878ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
116502026-03-24T21:06:05.878ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
116512026-03-24T21:06:05.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116522026-03-24T21:06:05.881ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
116532026-03-24T21:06:05.882ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "621c0bba-d2af-41a6-96da-eb2ab8c1bd29", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} })
116542026-03-24T21:06:05.883ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116552026-03-24T21:06:05.883ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "e0dcb20d-bdda-43c6-b0bc-2f21229548ae", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} })
116562026-03-24T21:06:05.883ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
116572026-03-24T21:06:05.883ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
116582026-03-24T21:06:05.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116592026-03-24T21:06:05.883ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 306.939246ms iteration = 10 status = {"errors":[],"probes_by_sled":{"b4293d4f-3308-4dfa-b261-a103215913e4":0}}
116602026-03-24T21:06:05.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
116612026-03-24T21:06:05.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116622026-03-24T21:06:05.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116632026-03-24T21:06:05.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116642026-03-24T21:06:05.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116652026-03-24T21:06:05.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116662026-03-24T21:06:05.888ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
116672026-03-24T21:06:05.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116682026-03-24T21:06:05.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
116692026-03-24T21:06:05.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116702026-03-24T21:06:05.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116712026-03-24T21:06:05.890ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116722026-03-24T21:06:05.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116732026-03-24T21:06:05.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
116742026-03-24T21:06:05.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116752026-03-24T21:06:05.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
116762026-03-24T21:06:05.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
116772026-03-24T21:06:05.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
116782026-03-24T21:06:05.894ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
116792026-03-24T21:06:05.894ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
116802026-03-24T21:06:05.895ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "617ec629-b73c-4fd2-982f-6d5eb19bde82", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} })
116812026-03-24T21:06:05.895ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 334.432438ms iteration = 28 status = {}
116822026-03-24T21:06:05.895ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Dependency
116832026-03-24T21:06:05.895ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
116842026-03-24T21:06:05.895ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116852026-03-24T21:06:05.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116862026-03-24T21:06:05.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116872026-03-24T21:06:05.897ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116882026-03-24T21:06:05.899ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
116892026-03-24T21:06:05.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
116902026-03-24T21:06:05.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
116912026-03-24T21:06:05.900ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
116922026-03-24T21:06:05.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
116932026-03-24T21:06:05.900ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
116942026-03-24T21:06:05.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
116952026-03-24T21:06:05.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
116962026-03-24T21:06:05.900ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
116972026-03-24T21:06:05.900ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116982026-03-24T21:06:05.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116992026-03-24T21:06:05.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117002026-03-24T21:06:05.902ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117012026-03-24T21:06:05.903ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117022026-03-24T21:06:05.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117032026-03-24T21:06:05.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
117042026-03-24T21:06:05.907ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
117052026-03-24T21:06:05.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117062026-03-24T21:06:05.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
117072026-03-24T21:06:05.908ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
117082026-03-24T21:06:05.908ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
117092026-03-24T21:06:05.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
117102026-03-24T21:06:05.909ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
117112026-03-24T21:06:05.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
117122026-03-24T21:06:05.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
117132026-03-24T21:06:05.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117142026-03-24T21:06:05.910ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117152026-03-24T21:06:05.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117162026-03-24T21:06:05.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117172026-03-24T21:06:05.913ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
117182026-03-24T21:06:05.913ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
117192026-03-24T21:06:05.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117202026-03-24T21:06:05.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
117212026-03-24T21:06:05.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
117222026-03-24T21:06:05.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
117232026-03-24T21:06:05.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117242026-03-24T21:06:05.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117252026-03-24T21:06:05.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
117262026-03-24T21:06:05.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
117272026-03-24T21:06:05.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117282026-03-24T21:06:05.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117292026-03-24T21:06:05.918ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
117302026-03-24T21:06:05.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117312026-03-24T21:06:05.919ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "39f0b4b0-93c6-483e-bb8b-5cbdc6c7e0df", "content-length": "691", "date": "Tue, 24 Mar 2026 21:06:05 GMT"} })
117322026-03-24T21:06:05.919ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
117332026-03-24T21:06:05.919ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
117342026-03-24T21:06:05.919ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
117352026-03-24T21:06:05.919ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 385.761672ms iteration = 19 status = {}
117362026-03-24T21:06:05.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117372026-03-24T21:06:05.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117382026-03-24T21:06:05.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117392026-03-24T21:06:05.922ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
117402026-03-24T21:06:05.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
117412026-03-24T21:06:05.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
117422026-03-24T21:06:05.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117432026-03-24T21:06:05.930ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
117442026-03-24T21:06:05.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117452026-03-24T21:06:05.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117462026-03-24T21:06:05.930ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
117472026-03-24T21:06:05.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
117482026-03-24T21:06:05.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
117492026-03-24T21:06:05.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117502026-03-24T21:06:05.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117512026-03-24T21:06:05.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
117522026-03-24T21:06:05.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
117532026-03-24T21:06:05.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
117542026-03-24T21:06:05.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
117552026-03-24T21:06:05.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117562026-03-24T21:06:05.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117572026-03-24T21:06:05.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117582026-03-24T21:06:05.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117592026-03-24T21:06:05.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117602026-03-24T21:06:05.941ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
117612026-03-24T21:06:05.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117622026-03-24T21:06:05.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117632026-03-24T21:06:05.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117642026-03-24T21:06:05.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117652026-03-24T21:06:05.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117662026-03-24T21:06:05.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117672026-03-24T21:06:05.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117682026-03-24T21:06:05.954ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
117692026-03-24T21:06:05.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117702026-03-24T21:06:05.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117712026-03-24T21:06:05.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117722026-03-24T21:06:05.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117732026-03-24T21:06:05.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117742026-03-24T21:06:05.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117752026-03-24T21:06:05.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117762026-03-24T21:06:05.972ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
117772026-03-24T21:06:05.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117782026-03-24T21:06:05.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117792026-03-24T21:06:05.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117802026-03-24T21:06:05.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117812026-03-24T21:06:05.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117822026-03-24T21:06:05.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117832026-03-24T21:06:05.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117842026-03-24T21:06:05.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117852026-03-24T21:06:05.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117862026-03-24T21:06:05.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117872026-03-24T21:06:05.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117882026-03-24T21:06:05.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117892026-03-24T21:06:05.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117902026-03-24T21:06:05.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117912026-03-24T21:06:05.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117922026-03-24T21:06:05.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117932026-03-24T21:06:06.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117942026-03-24T21:06:06.005ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
117952026-03-24T21:06:06.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117962026-03-24T21:06:06.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117972026-03-24T21:06:06.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117982026-03-24T21:06:06.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
117992026-03-24T21:06:06.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118002026-03-24T21:06:06.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118012026-03-24T21:06:06.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118022026-03-24T21:06:06.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118032026-03-24T21:06:06.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118042026-03-24T21:06:06.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118052026-03-24T21:06:06.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118062026-03-24T21:06:06.011ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118072026-03-24T21:06:06.016ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
118082026-03-24T21:06:06.017ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
118092026-03-24T21:06:06.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118102026-03-24T21:06:06.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118112026-03-24T21:06:06.018ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118122026-03-24T21:06:06.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118132026-03-24T21:06:06.020ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118142026-03-24T21:06:06.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118152026-03-24T21:06:06.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118162026-03-24T21:06:06.024ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
118172026-03-24T21:06:06.024ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118182026-03-24T21:06:06.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118192026-03-24T21:06:06.025ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
118202026-03-24T21:06:06.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118212026-03-24T21:06:06.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118222026-03-24T21:06:06.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118232026-03-24T21:06:06.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118242026-03-24T21:06:06.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118252026-03-24T21:06:06.030ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
118262026-03-24T21:06:06.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118272026-03-24T21:06:06.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118282026-03-24T21:06:06.031ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
118292026-03-24T21:06:06.031ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
118302026-03-24T21:06:06.032ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8835fd08-b85a-4902-8cd1-2566a7b0a5ad", "content-length": "372", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
118312026-03-24T21:06:06.032ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118322026-03-24T21:06:06.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118332026-03-24T21:06:06.033ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118342026-03-24T21:06:06.034ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
118352026-03-24T21:06:06.034ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
118362026-03-24T21:06:06.034ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
118372026-03-24T21:06:06.036ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
118382026-03-24T21:06:06.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118392026-03-24T21:06:06.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118402026-03-24T21:06:06.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118412026-03-24T21:06:06.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118422026-03-24T21:06:06.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118432026-03-24T21:06:06.045ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
118442026-03-24T21:06:06.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118452026-03-24T21:06:06.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118462026-03-24T21:06:06.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118472026-03-24T21:06:06.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118482026-03-24T21:06:06.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118492026-03-24T21:06:06.051ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:118 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
118502026-03-24T21:06:06.051ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 491.904414ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
118512026-03-24T21:06:06.051ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
118522026-03-24T21:06:06.051ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118532026-03-24T21:06:06.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118542026-03-24T21:06:06.099ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
118552026-03-24T21:06:06.099ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
118562026-03-24T21:06:06.099ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 8a703423-e691-4d6c-aacb-b2267807dd4b parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
118572026-03-24T21:06:06.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118582026-03-24T21:06:06.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118592026-03-24T21:06:06.102ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8903cfb2-93a9-456b-b4fc-0c953817eb3a", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
118602026-03-24T21:06:06.102ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
118612026-03-24T21:06:06.102ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
118622026-03-24T21:06:06.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
118632026-03-24T21:06:06.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118642026-03-24T21:06:06.107ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
118652026-03-24T21:06:06.107ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
118662026-03-24T21:06:06.107ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "0d3d5b4b-2c3e-4777-a2d3-b7c9e26af021", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
118672026-03-24T21:06:06.107ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118682026-03-24T21:06:06.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118692026-03-24T21:06:06.108ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
118702026-03-24T21:06:06.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118712026-03-24T21:06:06.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118722026-03-24T21:06:06.109ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
118732026-03-24T21:06:06.109ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
118742026-03-24T21:06:06.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118752026-03-24T21:06:06.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118762026-03-24T21:06:06.111ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118772026-03-24T21:06:06.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118782026-03-24T21:06:06.112ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
118792026-03-24T21:06:06.112ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
118802026-03-24T21:06:06.112ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 578.791617ms iteration = 24 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
118812026-03-24T21:06:06.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118822026-03-24T21:06:06.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118832026-03-24T21:06:06.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118842026-03-24T21:06:06.117ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
118852026-03-24T21:06:06.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
118862026-03-24T21:06:06.120ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
118872026-03-24T21:06:06.120ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
118882026-03-24T21:06:06.121ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "9bf4680c-b305-4713-b775-c969e4fb2183", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
118892026-03-24T21:06:06.121ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
118902026-03-24T21:06:06.121ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = Dns step id = Ensure
118912026-03-24T21:06:06.121ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
118922026-03-24T21:06:06.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118932026-03-24T21:06:06.122ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
118942026-03-24T21:06:06.122ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
118952026-03-24T21:06:06.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118962026-03-24T21:06:06.122ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7fbafe95-ee75-4511-81eb-0405d6434a89", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
118972026-03-24T21:06:06.122ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 227.311579ms iteration = 29 status = {}
118982026-03-24T21:06:06.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118992026-03-24T21:06:06.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119002026-03-24T21:06:06.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119012026-03-24T21:06:06.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
119022026-03-24T21:06:06.130ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
119032026-03-24T21:06:06.130ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
119042026-03-24T21:06:06.130ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
119052026-03-24T21:06:06.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119062026-03-24T21:06:06.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119072026-03-24T21:06:06.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119082026-03-24T21:06:06.135ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
119092026-03-24T21:06:06.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119102026-03-24T21:06:06.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119112026-03-24T21:06:06.156ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119122026-03-24T21:06:06.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119132026-03-24T21:06:06.159ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119142026-03-24T21:06:06.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119152026-03-24T21:06:06.161ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119162026-03-24T21:06:06.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119172026-03-24T21:06:06.163ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 629.013081ms iteration = 19 status = {}
119182026-03-24T21:06:06.183ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
119192026-03-24T21:06:06.183ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
119202026-03-24T21:06:06.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119212026-03-24T21:06:06.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119222026-03-24T21:06:06.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119232026-03-24T21:06:06.189ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
119242026-03-24T21:06:06.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119252026-03-24T21:06:06.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119262026-03-24T21:06:06.191ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
119272026-03-24T21:06:06.191ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
119282026-03-24T21:06:06.191ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
119292026-03-24T21:06:06.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119302026-03-24T21:06:06.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119312026-03-24T21:06:06.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119322026-03-24T21:06:06.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119332026-03-24T21:06:06.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119342026-03-24T21:06:06.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
119352026-03-24T21:06:06.245ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
119362026-03-24T21:06:06.245ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
119372026-03-24T21:06:06.245ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
119382026-03-24T21:06:06.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119392026-03-24T21:06:06.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119402026-03-24T21:06:06.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119412026-03-24T21:06:06.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
119422026-03-24T21:06:06.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119432026-03-24T21:06:06.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119442026-03-24T21:06:06.253ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
119452026-03-24T21:06:06.253ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
119462026-03-24T21:06:06.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119472026-03-24T21:06:06.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119482026-03-24T21:06:06.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119492026-03-24T21:06:06.258ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
119502026-03-24T21:06:06.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119512026-03-24T21:06:06.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119522026-03-24T21:06:06.260ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
119532026-03-24T21:06:06.260ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
119542026-03-24T21:06:06.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119552026-03-24T21:06:06.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119562026-03-24T21:06:06.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119572026-03-24T21:06:06.264ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
119582026-03-24T21:06:06.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119592026-03-24T21:06:06.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119602026-03-24T21:06:06.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119612026-03-24T21:06:06.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119622026-03-24T21:06:06.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119632026-03-24T21:06:06.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119642026-03-24T21:06:06.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119652026-03-24T21:06:06.272ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
119662026-03-24T21:06:06.273ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
119672026-03-24T21:06:06.273ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
119682026-03-24T21:06:06.273ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = OmicronZones step id = Cleanup
119692026-03-24T21:06:06.273ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = OmicronZones step id = Cleanup
119702026-03-24T21:06:06.273ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = PhysicalDisks step id = Cleanup
119712026-03-24T21:06:06.274ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = Clickhouse step id = Ensure
119722026-03-24T21:06:06.274ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = Clickhouse step id = Ensure
119732026-03-24T21:06:06.274ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
119742026-03-24T21:06:06.279ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "81fb7823-82f7-49f9-ab1d-8d21eec1162a", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
119752026-03-24T21:06:06.279ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = SupportBundles step id = Cleanup
119762026-03-24T21:06:06.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119772026-03-24T21:06:06.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119782026-03-24T21:06:06.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119792026-03-24T21:06:06.282ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
119802026-03-24T21:06:06.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119812026-03-24T21:06:06.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119822026-03-24T21:06:06.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119832026-03-24T21:06:06.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119842026-03-24T21:06:06.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119852026-03-24T21:06:06.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
119862026-03-24T21:06:06.334ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = OmicronZones step id = Cleanup
119872026-03-24T21:06:06.334ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
119882026-03-24T21:06:06.334ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
119892026-03-24T21:06:06.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119902026-03-24T21:06:06.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119912026-03-24T21:06:06.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119922026-03-24T21:06:06.337ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
119932026-03-24T21:06:06.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119942026-03-24T21:06:06.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119952026-03-24T21:06:06.377ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
119962026-03-24T21:06:06.377ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9e1c09c1-3960-4ecf-a904-19e137699ab3 (blueprint), false)
119972026-03-24T21:06:06.377ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = Cockroach step id = Ensure
119982026-03-24T21:06:06.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119992026-03-24T21:06:06.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
120002026-03-24T21:06:06.411ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
120012026-03-24T21:06:06.411ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f308010d-74ae-4cd8-a06d-d59b24d30f1e step component = MgsUpdates step id = Ensure
120022026-03-24T21:06:06.411ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 12 reason = Dependency
120032026-03-24T21:06:06.411ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 29 reason = Dependency
120042026-03-24T21:06:06.411ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
120052026-03-24T21:06:06.411ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120062026-03-24T21:06:06.411ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 890.07527ms iteration = 11 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3831,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":89656958,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":89656918,"secs":0}},"event_index":1,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":89672549,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":98229769,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":98229719,"secs":0}},"event_index":2,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":187919418,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":147716203,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":147716163,"secs":0}},"event_index":3,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":335651611,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4790310,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4790279,"secs":0}},"event_index":4,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":340482781,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":259803405,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":259803355,"secs":0}},"event_index":5,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":600295927,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":152137953,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":152137903,"secs":0}},"event_index":6,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":752443440,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5240,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5210,"secs":0}},"event_index":7,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":752460300,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2641,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2610,"secs":0}},"event_index":8,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":752465170,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3371,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3331,"secs":0}},"event_index":9,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":752470361,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1790,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1760,"secs":0}},"event_index":10,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":752473681,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5463047,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":5463007,"secs":0}},"event_index":11,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":757937528,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":54561401,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":54561361,"secs":0}},"event_index":12,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":812510289,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":43849753,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":43849713,"secs":0}},"event_index":13,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":856383443,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":33355288,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":33355258,"secs":0}},"event_index":14,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":889749151,"secs":0}},{"data":{"attempt_elapsed":{"nanos":5131,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5091,"secs":0}},"event_index":15,"execution_id":"61f3c64a-4e26-447b-9b74-86605a902064","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":889761312,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3"}
120072026-03-24T21:06:06.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
120082026-03-24T21:06:06.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
120092026-03-24T21:06:06.413ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120102026-03-24T21:06:06.413ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120112026-03-24T21:06:06.416ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
120122026-03-24T21:06:06.416ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
120132026-03-24T21:06:06.416ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
120142026-03-24T21:06:06.416ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120152026-03-24T21:06:06.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
120162026-03-24T21:06:06.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
120172026-03-24T21:06:06.458ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
120182026-03-24T21:06:06.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
120192026-03-24T21:06:06.459ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120202026-03-24T21:06:06.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
120212026-03-24T21:06:06.462ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
120222026-03-24T21:06:06.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
120232026-03-24T21:06:06.464ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
120242026-03-24T21:06:06.464ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 52.884406ms iteration = 29 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
120252026-03-24T21:06:06.496ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(bfe73e65-b1ea-44ad-b413-8bbb37fec3ef, 0), (f2b002cd-8c10-43b6-b761-bd26c043dd67, 0), (fd570abe-50d6-4436-9995-41c34b8b6854, 0), (536e3610-cdc3-4254-98ba-07880214c42c, 0)] collection_id = bfe73e65-b1ea-44ad-b413-8bbb37fec3ef
120262026-03-24T21:06:06.496ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120272026-03-24T21:06:06.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
120282026-03-24T21:06:06.714ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = bfe73e65-b1ea-44ad-b413-8bbb37fec3ef file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
120292026-03-24T21:06:06.714ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120302026-03-24T21:06:06.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
120312026-03-24T21:06:06.717ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(f2b002cd-8c10-43b6-b761-bd26c043dd67, 0), (fd570abe-50d6-4436-9995-41c34b8b6854, 0), (536e3610-cdc3-4254-98ba-07880214c42c, 0)]
120322026-03-24T21:06:06.717ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1352, tv_nsec: 931445969 } })
120332026-03-24T21:06:06.719ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1345db7a-abc1-4a4c-a49c-e6c56102e521.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1920db37-87f5-4a96-b3ef-1db2eb59e9a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6b02d2eb-4373-4f12-8efc-4c0e3f4fa77b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("709c4704-f768-4640-a198-a23621ed6e11.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1352, tv_nsec: 933561346 } })
120342026-03-24T21:06:06.720ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
120352026-03-24T21:06:06.720ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
120362026-03-24T21:06:06.720ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
120372026-03-24T21:06:06.720ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8459ce0a-75f2-40a2-a310-6a29d96fce4c", "content-length": "476", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120382026-03-24T21:06:06.721ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
120392026-03-24T21:06:06.721ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e91ec251-bb82-4797-9a36-3811a0fa12b1", "content-length": "734", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120402026-03-24T21:06:06.721ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
120412026-03-24T21:06:06.721ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5afa104f-6673-4f4e-a226-cf4ebf26ead8", "content-length": "111", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120422026-03-24T21:06:06.721ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
120432026-03-24T21:06:06.722ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb433a4d-bc8a-4d54-be05-132f9bb813ca", "content-length": "111", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120442026-03-24T21:06:06.722ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
120452026-03-24T21:06:06.722ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e569204c-ac10-4eec-83ed-873a47a388fc", "content-length": "163", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120462026-03-24T21:06:06.722ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
120472026-03-24T21:06:06.723ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b9f1de0-b3f6-4e2f-b05f-981635103445", "content-length": "163", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120482026-03-24T21:06:06.723ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
120492026-03-24T21:06:06.723ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b5752ce-c034-4f9b-bc50-6eb1a8b3bda8", "content-length": "166", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120502026-03-24T21:06:06.723ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
120512026-03-24T21:06:06.723ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "75465cc0-077b-41cb-a9b5-ab682e442b78", "content-length": "166", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120522026-03-24T21:06:06.723ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
120532026-03-24T21:06:06.724ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60745c53-e5ab-4a5a-ac61-c4606c38193b", "content-length": "702", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120542026-03-24T21:06:06.724ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
120552026-03-24T21:06:06.724ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "645fda08-2f05-4aa4-86a8-305984782c30", "content-length": "727", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120562026-03-24T21:06:06.724ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
120572026-03-24T21:06:06.724ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "68267c4d-659d-4779-b083-3d1a1cc8adec", "content-length": "729", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120582026-03-24T21:06:06.724ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
120592026-03-24T21:06:06.724ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cb120e9-f2ab-436c-b3a6-b496607c9650", "content-length": "728", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120602026-03-24T21:06:06.724ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
120612026-03-24T21:06:06.724ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6fcbcf1-d143-4270-9229-1304fe17f960", "content-length": "722", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120622026-03-24T21:06:06.725ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
120632026-03-24T21:06:06.725ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c24196b-9219-46df-a84a-37086267509f", "content-length": "10", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120642026-03-24T21:06:06.725ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
120652026-03-24T21:06:06.725ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "390b9a54-2b9c-4409-80db-76ebacfaaeef", "content-length": "144", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120662026-03-24T21:06:06.725ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
120672026-03-24T21:06:06.725ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b57c102-b45a-47a1-ab4a-42060fc40314", "content-length": "147", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120682026-03-24T21:06:06.725ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
120692026-03-24T21:06:06.725ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9049d575-8b70-48dc-9461-c4c0cc7dddb2", "content-length": "109", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120702026-03-24T21:06:06.725ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
120712026-03-24T21:06:06.726ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb024db9-9d6e-4a8f-a07e-c16689e02e1c", "content-length": "109", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120722026-03-24T21:06:06.726ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
120732026-03-24T21:06:06.726ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc01497a-6cc3-4272-8eb5-54b20d339999", "content-length": "163", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120742026-03-24T21:06:06.726ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
120752026-03-24T21:06:06.726ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "287b477a-4008-4946-8bd8-344157c63159", "content-length": "163", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120762026-03-24T21:06:06.727ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
120772026-03-24T21:06:06.727ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9e2fc45-e224-47a1-ba84-45b9ded8a343", "content-length": "166", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120782026-03-24T21:06:06.727ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
120792026-03-24T21:06:06.727ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "948cf336-12e4-43fa-910d-a5a33bac0f8f", "content-length": "166", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120802026-03-24T21:06:06.727ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
120812026-03-24T21:06:06.727ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3ce0784-5f6c-45fa-b15f-6dbc1b6b995f", "content-length": "702", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120822026-03-24T21:06:06.727ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
120832026-03-24T21:06:06.728ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e954352c-922f-467d-9f05-cb1a9128317f", "content-length": "727", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120842026-03-24T21:06:06.728ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
120852026-03-24T21:06:06.728ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d251486-3bef-4dab-8583-974db2398537", "content-length": "729", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120862026-03-24T21:06:06.728ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
120872026-03-24T21:06:06.728ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b1a4e7a-a8af-424e-aef3-b8230370dc1a", "content-length": "728", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
120882026-03-24T21:06:06.728ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120892026-03-24T21:06:06.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
120902026-03-24T21:06:06.730ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120912026-03-24T21:06:06.732ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
120922026-03-24T21:06:06.732ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120932026-03-24T21:06:06.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
120942026-03-24T21:06:06.733ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120952026-03-24T21:06:06.736ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
120962026-03-24T21:06:06.736ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120972026-03-24T21:06:06.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
120982026-03-24T21:06:06.742ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
120992026-03-24T21:06:06.742ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
121002026-03-24T21:06:06.835ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "10c3a511-388c-44fd-ba88-646a6a2d5c84", "content-length": "110672", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121012026-03-24T21:06:06.835ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
121022026-03-24T21:06:06.835ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
121032026-03-24T21:06:06.835ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
121042026-03-24T21:06:06.835ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
121052026-03-24T21:06:06.835ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
121062026-03-24T21:06:06.835ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
121072026-03-24T21:06:06.835ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
121082026-03-24T21:06:06.835ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
121092026-03-24T21:06:06.836ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c892186-b861-412b-a393-98916b0740b5", "content-length": "64", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121102026-03-24T21:06:06.836ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f538369-257f-4d8d-a501-e5543d0a3a8e", "content-length": "64", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121112026-03-24T21:06:06.836ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
121122026-03-24T21:06:06.836ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
121132026-03-24T21:06:06.836ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "caa90193-0708-4924-a4c0-2e60805de410", "content-length": "64", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121142026-03-24T21:06:06.836ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
121152026-03-24T21:06:06.836ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcb24145-659e-4956-b076-888211e8c3f4", "content-length": "64", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121162026-03-24T21:06:06.836ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d19bebf4-93e3-4562-9e89-ac498a8cca41", "content-length": "64", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121172026-03-24T21:06:06.836ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
121182026-03-24T21:06:06.836ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
121192026-03-24T21:06:06.843ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c562da8a-83e4-4f62-a8e8-87117c45fb36", "content-length": "409905", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121202026-03-24T21:06:06.844ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ec544f1-01c3-42ce-9d03-41a9999a74fb", "content-length": "399997", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121212026-03-24T21:06:06.844ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "2496d941-3110-4fda-9323-ede479a256fe", "content-length": "401686", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121222026-03-24T21:06:06.845ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "4388d4dd-93ec-4145-87ca-389ff16b31d9", "content-length": "430307", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121232026-03-24T21:06:06.846ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "087a085a-7b3c-4fc7-b194-edd938ac8f07", "content-length": "404228", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121242026-03-24T21:06:06.849ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
121252026-03-24T21:06:06.849ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
121262026-03-24T21:06:06.849ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
121272026-03-24T21:06:06.849ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
121282026-03-24T21:06:06.849ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
121292026-03-24T21:06:06.850ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 (omicron_zone)
121302026-03-24T21:06:06.850ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
121312026-03-24T21:06:06.853ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a45dceb-b663-4dcc-a8c1-4bb8b0fd65b6", "content-length": "127", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121322026-03-24T21:06:06.853ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
121332026-03-24T21:06:06.855ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
121342026-03-24T21:06:06.855ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
121352026-03-24T21:06:06.856ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa31bd10-b8b8-4d9e-a194-95cd61bca859", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121362026-03-24T21:06:06.856ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
121372026-03-24T21:06:06.856ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
121382026-03-24T21:06:06.856ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
121392026-03-24T21:06:06.856ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cb592bd-d83c-4935-a65a-b07acd982302", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121402026-03-24T21:06:06.857ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
121412026-03-24T21:06:06.857ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
121422026-03-24T21:06:06.857ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
121432026-03-24T21:06:06.857ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0a45029-6775-444f-847d-e84b18ddd461", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:06:06 GMT"} })
121442026-03-24T21:06:06.857ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
121452026-03-24T21:06:06.857ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
121462026-03-24T21:06:06.857ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
121472026-03-24T21:06:06.857ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
121482026-03-24T21:06:06.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
121492026-03-24T21:06:06.859ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121502026-03-24T21:06:06.861ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
121512026-03-24T21:06:06.861ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
121522026-03-24T21:06:06.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
121532026-03-24T21:06:07.084ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = df496995-f9d8-4235-b1db-c1cf67df3a52 file = nexus/db-queries/src/db/datastore/inventory.rs:1757
121542026-03-24T21:06:07.084ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = df496995-f9d8-4235-b1db-c1cf67df3a52 time_started = 2026-03-24 21:06:06.720501 UTC
121552026-03-24T21:06:07.084ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 49 reason = Dependency
121562026-03-24T21:06:07.084ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
121572026-03-24T21:06:07.084ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 672.999478ms iteration = 12 status = {"collection_id":"df496995-f9d8-4235-b1db-c1cf67df3a52","time_done":"2026-03-24 21:06:06.857717 UTC","time_started":"2026-03-24 21:06:06.720501 UTC"}
121582026-03-24T21:06:07.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
121592026-03-24T21:06:07.086ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121602026-03-24T21:06:07.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
121612026-03-24T21:06:07.089ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
121622026-03-24T21:06:07.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
121632026-03-24T21:06:07.091ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
121642026-03-24T21:06:07.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
121652026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Dependency
121662026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 58.827086ms iteration = 49 status = {"Loaded":{"collection_id":"df496995-f9d8-4235-b1db-c1cf67df3a52","time_loaded":"2026-03-24T21:06:07.084316134Z","time_started":"2026-03-24T21:06:06.720501Z"}}
121672026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 23 reason = Dependency
121682026-03-24T21:06:07.143ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
121692026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 25 reason = Dependency
121702026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
121712026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 13.67\u{b5}s iteration = 23 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
121722026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
121732026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 15 reason = Dependency
121742026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
121752026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
121762026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
121772026-03-24T21:06:07.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
121782026-03-24T21:06:07.144ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121792026-03-24T21:06:07.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121802026-03-24T21:06:07.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121812026-03-24T21:06:07.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
121822026-03-24T21:06:07.148ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
121832026-03-24T21:06:07.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
121842026-03-24T21:06:07.148ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
121852026-03-24T21:06:07.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
121862026-03-24T21:06:07.148ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
121872026-03-24T21:06:07.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
121882026-03-24T21:06:07.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
121892026-03-24T21:06:07.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
121902026-03-24T21:06:07.149ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
121912026-03-24T21:06:07.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121922026-03-24T21:06:07.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121932026-03-24T21:06:07.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
121942026-03-24T21:06:07.151ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121952026-03-24T21:06:07.152ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
121962026-03-24T21:06:07.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
121972026-03-24T21:06:07.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
121982026-03-24T21:06:07.153ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
121992026-03-24T21:06:07.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
122002026-03-24T21:06:07.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
122012026-03-24T21:06:07.153ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
122022026-03-24T21:06:07.153ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
122032026-03-24T21:06:07.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
122042026-03-24T21:06:07.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
122052026-03-24T21:06:07.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
122062026-03-24T21:06:07.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122072026-03-24T21:06:07.159ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
122082026-03-24T21:06:07.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
122092026-03-24T21:06:07.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
122102026-03-24T21:06:07.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122112026-03-24T21:06:07.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
122122026-03-24T21:06:07.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
122132026-03-24T21:06:07.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
122142026-03-24T21:06:07.165ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
122152026-03-24T21:06:07.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
122162026-03-24T21:06:07.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
122172026-03-24T21:06:07.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122182026-03-24T21:06:07.169ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
122192026-03-24T21:06:07.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
122202026-03-24T21:06:07.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
122212026-03-24T21:06:07.212ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 69.536575ms iteration = 32 status = {"physical_disks_added":0}
122222026-03-24T21:06:07.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122232026-03-24T21:06:07.220ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
122242026-03-24T21:06:07.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
122252026-03-24T21:06:07.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
122262026-03-24T21:06:07.223ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
122272026-03-24T21:06:07.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
122282026-03-24T21:06:07.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
122292026-03-24T21:06:07.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122302026-03-24T21:06:07.227ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
122312026-03-24T21:06:07.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
122322026-03-24T21:06:07.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
122332026-03-24T21:06:07.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122342026-03-24T21:06:07.231ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
122352026-03-24T21:06:07.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
122362026-03-24T21:06:07.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
122372026-03-24T21:06:07.234ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
122382026-03-24T21:06:07.234ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 91.155818ms iteration = 15 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"df496995-f9d8-4235-b1db-c1cf67df3a52","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
122392026-03-24T21:06:07.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122402026-03-24T21:06:07.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122412026-03-24T21:06:07.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122422026-03-24T21:06:07.249ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
122432026-03-24T21:06:07.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122442026-03-24T21:06:07.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122452026-03-24T21:06:07.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122462026-03-24T21:06:07.252ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
122472026-03-24T21:06:07.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122482026-03-24T21:06:07.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122492026-03-24T21:06:07.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122502026-03-24T21:06:07.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122512026-03-24T21:06:07.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122522026-03-24T21:06:07.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122532026-03-24T21:06:07.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122542026-03-24T21:06:07.262ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
122552026-03-24T21:06:07.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122562026-03-24T21:06:07.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122572026-03-24T21:06:07.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122582026-03-24T21:06:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122592026-03-24T21:06:07.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122602026-03-24T21:06:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
122612026-03-24T21:06:07.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122622026-03-24T21:06:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122632026-03-24T21:06:07.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122642026-03-24T21:06:07.272ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
122652026-03-24T21:06:07.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122662026-03-24T21:06:07.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122672026-03-24T21:06:07.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122682026-03-24T21:06:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122692026-03-24T21:06:07.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122702026-03-24T21:06:07.278ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
122712026-03-24T21:06:07.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122722026-03-24T21:06:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122732026-03-24T21:06:07.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122742026-03-24T21:06:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122752026-03-24T21:06:07.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122762026-03-24T21:06:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
122772026-03-24T21:06:07.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122782026-03-24T21:06:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122792026-03-24T21:06:07.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122802026-03-24T21:06:07.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122812026-03-24T21:06:07.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122822026-03-24T21:06:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
122832026-03-24T21:06:07.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122842026-03-24T21:06:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122852026-03-24T21:06:07.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122862026-03-24T21:06:07.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122872026-03-24T21:06:07.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122882026-03-24T21:06:07.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122892026-03-24T21:06:07.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122902026-03-24T21:06:07.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
122912026-03-24T21:06:07.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122922026-03-24T21:06:07.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122932026-03-24T21:06:07.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122942026-03-24T21:06:07.318ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
122952026-03-24T21:06:07.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122962026-03-24T21:06:07.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122972026-03-24T21:06:07.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122982026-03-24T21:06:07.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122992026-03-24T21:06:07.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123002026-03-24T21:06:07.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123012026-03-24T21:06:07.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123022026-03-24T21:06:07.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
123032026-03-24T21:06:07.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123042026-03-24T21:06:07.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123052026-03-24T21:06:07.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123062026-03-24T21:06:07.335ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
123072026-03-24T21:06:07.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123082026-03-24T21:06:07.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123092026-03-24T21:06:07.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123102026-03-24T21:06:07.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123112026-03-24T21:06:07.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123122026-03-24T21:06:07.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123132026-03-24T21:06:07.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123142026-03-24T21:06:07.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
123152026-03-24T21:06:07.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123162026-03-24T21:06:07.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123172026-03-24T21:06:07.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123182026-03-24T21:06:07.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123192026-03-24T21:06:07.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123202026-03-24T21:06:07.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
123212026-03-24T21:06:07.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123222026-03-24T21:06:07.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123232026-03-24T21:06:07.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123242026-03-24T21:06:07.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123252026-03-24T21:06:07.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123262026-03-24T21:06:07.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
123272026-03-24T21:06:07.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123282026-03-24T21:06:07.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123292026-03-24T21:06:07.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123302026-03-24T21:06:07.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123312026-03-24T21:06:07.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123322026-03-24T21:06:07.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
123332026-03-24T21:06:07.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123342026-03-24T21:06:07.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123352026-03-24T21:06:07.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123362026-03-24T21:06:07.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123372026-03-24T21:06:07.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123382026-03-24T21:06:07.370ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
123392026-03-24T21:06:07.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123402026-03-24T21:06:07.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123412026-03-24T21:06:07.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123422026-03-24T21:06:07.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123432026-03-24T21:06:07.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123442026-03-24T21:06:07.376ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
123452026-03-24T21:06:07.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123462026-03-24T21:06:07.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123472026-03-24T21:06:07.419ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
123482026-03-24T21:06:07.419ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
123492026-03-24T21:06:07.419ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 743c1a2d-0cee-4a9d-accc-19a15a42134a parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
123502026-03-24T21:06:07.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123512026-03-24T21:06:07.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123522026-03-24T21:06:07.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123532026-03-24T21:06:07.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
123542026-03-24T21:06:07.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123552026-03-24T21:06:07.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123562026-03-24T21:06:07.427ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
123572026-03-24T21:06:07.427ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
123582026-03-24T21:06:07.427ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 284.285675ms iteration = 25 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
123592026-03-24T21:06:08.552ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = e42417b1-401c-48f0-ad29-28480fefbae5 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
123602026-03-24T21:06:08.552ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = e42417b1-401c-48f0-ad29-28480fefbae5 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
123612026-03-24T21:06:08.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = e42417b1-401c-48f0-ad29-28480fefbae5 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
123622026-03-24T21:06:08.554ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2516 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = e42417b1-401c-48f0-ad29-28480fefbae5 response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
123632026-03-24T21:06:08.554ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 4ef784ce-5dda-4297-b7ab-a266f997f0a3 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
123642026-03-24T21:06:08.554ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 4ef784ce-5dda-4297-b7ab-a266f997f0a3 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
123652026-03-24T21:06:08.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 4ef784ce-5dda-4297-b7ab-a266f997f0a3 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
123662026-03-24T21:06:08.557ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2706 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 4ef784ce-5dda-4297-b7ab-a266f997f0a3 response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
123672026-03-24T21:06:08.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 368 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = b088ca1c-dc19-4e4d-8d77-ac4cc1f45b47 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
123682026-03-24T21:06:10.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 110 reason = Timeout
123692026-03-24T21:06:10.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
123702026-03-24T21:06:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
123712026-03-24T21:06:10.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123722026-03-24T21:06:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
123732026-03-24T21:06:10.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
123742026-03-24T21:06:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
123752026-03-24T21:06:10.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
123762026-03-24T21:06:10.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.186041ms iteration = 110 status = {"config_updated":false}
123772026-03-24T21:06:15.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
123782026-03-24T21:06:15.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 111 reason = Timeout
123792026-03-24T21:06:15.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123802026-03-24T21:06:15.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
123812026-03-24T21:06:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
123822026-03-24T21:06:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
123832026-03-24T21:06:15.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123842026-03-24T21:06:15.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123852026-03-24T21:06:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
123862026-03-24T21:06:15.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
123872026-03-24T21:06:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
123882026-03-24T21:06:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
123892026-03-24T21:06:15.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123902026-03-24T21:06:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
123912026-03-24T21:06:15.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
123922026-03-24T21:06:15.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.436251ms iteration = 111 status = {"config_updated":false}
123932026-03-24T21:06:15.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123942026-03-24T21:06:15.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
123952026-03-24T21:06:15.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123962026-03-24T21:06:15.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
123972026-03-24T21:06:15.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
123982026-03-24T21:06:15.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
123992026-03-24T21:06:15.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
124002026-03-24T21:06:15.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
124012026-03-24T21:06:15.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
124022026-03-24T21:06:15.563ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
124032026-03-24T21:06:15.563ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.821161ms iteration = 56 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
124042026-03-24T21:06:15.605ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
124052026-03-24T21:06:15.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
124062026-03-24T21:06:15.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 84.748495ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
124072026-03-24T21:06:18.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 378 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 4017bfe3-d4e1-462b-86dc-f36ff98d36ee response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
124082026-03-24T21:06:20.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 50 reason = Timeout
124092026-03-24T21:06:20.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
124102026-03-24T21:06:20.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 112 reason = Timeout
124112026-03-24T21:06:20.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
124122026-03-24T21:06:20.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
124132026-03-24T21:06:20.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
124142026-03-24T21:06:20.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124152026-03-24T21:06:20.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124162026-03-24T21:06:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
124172026-03-24T21:06:20.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
124182026-03-24T21:06:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
124192026-03-24T21:06:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
124202026-03-24T21:06:20.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
124212026-03-24T21:06:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
124222026-03-24T21:06:20.540ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = df496995-f9d8-4235-b1db-c1cf67df3a52 original_time_started = 2026-03-24 21:06:06.720501 UTC
124232026-03-24T21:06:20.540ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.800551ms iteration = 50 status = {"Loaded":{"collection_id":"df496995-f9d8-4235-b1db-c1cf67df3a52","time_loaded":"2026-03-24T21:06:20.532904902Z","time_started":"2026-03-24T21:06:06.720501Z"}}
124242026-03-24T21:06:20.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
124252026-03-24T21:06:20.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.274803ms iteration = 112 status = {"config_updated":false}
124262026-03-24T21:06:20.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 38 reason = Timeout
124272026-03-24T21:06:20.575ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 38 reason = Timeout
124282026-03-24T21:06:20.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
124292026-03-24T21:06:20.575ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
124302026-03-24T21:06:20.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
124312026-03-24T21:06:20.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
124322026-03-24T21:06:20.576ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
124332026-03-24T21:06:20.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
124342026-03-24T21:06:20.577ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124352026-03-24T21:06:20.578ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124362026-03-24T21:06:20.579ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
124372026-03-24T21:06:20.580ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
124382026-03-24T21:06:20.580ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
124392026-03-24T21:06:20.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
124402026-03-24T21:06:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
124412026-03-24T21:06:20.582ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.416333ms iteration = 38 status = "NoSitrep"
124422026-03-24T21:06:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
124432026-03-24T21:06:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
124442026-03-24T21:06:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
124452026-03-24T21:06:20.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124462026-03-24T21:06:20.585ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
124472026-03-24T21:06:20.589ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
124482026-03-24T21:06:20.589ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
124492026-03-24T21:06:20.589ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.561776ms iteration = 38 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
124502026-03-24T21:06:25.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
124512026-03-24T21:06:25.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 113 reason = Timeout
124522026-03-24T21:06:25.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
124532026-03-24T21:06:25.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
124542026-03-24T21:06:25.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
124552026-03-24T21:06:25.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
124562026-03-24T21:06:25.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124572026-03-24T21:06:25.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124582026-03-24T21:06:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
124592026-03-24T21:06:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
124602026-03-24T21:06:25.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
124612026-03-24T21:06:25.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
124622026-03-24T21:06:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
124632026-03-24T21:06:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
124642026-03-24T21:06:25.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
124652026-03-24T21:06:25.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.233719ms iteration = 113 status = {"config_updated":false}
124662026-03-24T21:06:25.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
124672026-03-24T21:06:25.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
124682026-03-24T21:06:25.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124692026-03-24T21:06:25.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
124702026-03-24T21:06:25.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
124712026-03-24T21:06:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
124722026-03-24T21:06:25.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
124732026-03-24T21:06:25.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
124742026-03-24T21:06:25.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
124752026-03-24T21:06:25.562ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
124762026-03-24T21:06:25.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
124772026-03-24T21:06:25.562ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
124782026-03-24T21:06:25.562ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.450224ms iteration = 57 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
124792026-03-24T21:06:25.608ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 47.622554ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
124802026-03-24T21:06:28.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 600 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 28030a52-6a81-48dd-8c51-2bb16b7928cb response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
124812026-03-24T21:06:30.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 114 reason = Timeout
124822026-03-24T21:06:30.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
124832026-03-24T21:06:30.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
124842026-03-24T21:06:30.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124852026-03-24T21:06:30.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
124862026-03-24T21:06:30.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
124872026-03-24T21:06:30.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
124882026-03-24T21:06:30.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
124892026-03-24T21:06:30.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.4018ms iteration = 114 status = {"config_updated":false}
124902026-03-24T21:06:35.521ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 20 reason = Timeout
124912026-03-24T21:06:35.521ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
124922026-03-24T21:06:35.521ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
124932026-03-24T21:06:35.521ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
124942026-03-24T21:06:35.521ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
124952026-03-24T21:06:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
124962026-03-24T21:06:35.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
124972026-03-24T21:06:35.526ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1381, tv_nsec: 740299011 } })
124982026-03-24T21:06:35.527ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
124992026-03-24T21:06:35.527ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1381, tv_nsec: 741178755 } })
125002026-03-24T21:06:35.528ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1381, tv_nsec: 742003077 } })
125012026-03-24T21:06:35.529ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
125022026-03-24T21:06:35.529ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
125032026-03-24T21:06:35.530ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb6f0494-7d8a-4ea3-a067-b420bd4baa92", "content-length": "26", "date": "Tue, 24 Mar 2026 21:06:35 GMT"} })
125042026-03-24T21:06:35.530ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
125052026-03-24T21:06:35.530ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
125062026-03-24T21:06:35.530ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
125072026-03-24T21:06:35.530ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 9.419674ms iteration = 20 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
125082026-03-24T21:06:35.531ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1381, tv_nsec: 744767712 } })
125092026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
125102026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
125112026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
125122026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125132026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
125142026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 51 reason = Timeout
125152026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125162026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 115 reason = Timeout
125172026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
125182026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
125192026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
125202026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 30 reason = Timeout
125212026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
125222026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
125232026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
125242026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
125252026-03-24T21:06:35.532ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 11.242563ms iteration = 20 status = {}
125262026-03-24T21:06:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
125272026-03-24T21:06:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125282026-03-24T21:06:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
125292026-03-24T21:06:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
125302026-03-24T21:06:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
125312026-03-24T21:06:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
125322026-03-24T21:06:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
125332026-03-24T21:06:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
125342026-03-24T21:06:35.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125352026-03-24T21:06:35.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125362026-03-24T21:06:35.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125372026-03-24T21:06:35.535ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125382026-03-24T21:06:35.536ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 14.507451ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
125392026-03-24T21:06:35.536ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125402026-03-24T21:06:35.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125412026-03-24T21:06:35.536ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
125422026-03-24T21:06:35.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
125432026-03-24T21:06:35.539ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125442026-03-24T21:06:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
125452026-03-24T21:06:35.540ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
125462026-03-24T21:06:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
125472026-03-24T21:06:35.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125482026-03-24T21:06:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
125492026-03-24T21:06:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
125502026-03-24T21:06:35.541ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
125512026-03-24T21:06:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125522026-03-24T21:06:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
125532026-03-24T21:06:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
125542026-03-24T21:06:35.541ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
125552026-03-24T21:06:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
125562026-03-24T21:06:35.541ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
125572026-03-24T21:06:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
125582026-03-24T21:06:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
125592026-03-24T21:06:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
125602026-03-24T21:06:35.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125612026-03-24T21:06:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
125622026-03-24T21:06:35.543ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
125632026-03-24T21:06:35.544ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = df496995-f9d8-4235-b1db-c1cf67df3a52 original_time_started = 2026-03-24 21:06:06.720501 UTC
125642026-03-24T21:06:35.544ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 11.668155ms iteration = 51 status = {"Loaded":{"collection_id":"df496995-f9d8-4235-b1db-c1cf67df3a52","time_loaded":"2026-03-24T21:06:35.532546921Z","time_started":"2026-03-24T21:06:06.720501Z"}}
125652026-03-24T21:06:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
125662026-03-24T21:06:35.544ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
125672026-03-24T21:06:35.544ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 11.742317ms iteration = 115 status = {"config_updated":false}
125682026-03-24T21:06:35.544ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1381, tv_nsec: 758245405 } })
125692026-03-24T21:06:35.544ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
125702026-03-24T21:06:35.545ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1381, tv_nsec: 758997806 } })
125712026-03-24T21:06:35.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
125722026-03-24T21:06:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
125732026-03-24T21:06:35.545ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
125742026-03-24T21:06:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
125752026-03-24T21:06:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
125762026-03-24T21:06:35.546ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125772026-03-24T21:06:35.546ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
125782026-03-24T21:06:35.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
125792026-03-24T21:06:35.546ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
125802026-03-24T21:06:35.546ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
125812026-03-24T21:06:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
125822026-03-24T21:06:35.547ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7aefed6b-8d3c-4ec2-af0f-1d6f684f9264", "content-length": "26", "date": "Tue, 24 Mar 2026 21:06:35 GMT"} })
125832026-03-24T21:06:35.547ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
125842026-03-24T21:06:35.547ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
125852026-03-24T21:06:35.547ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
125862026-03-24T21:06:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
125872026-03-24T21:06:35.548ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1381, tv_nsec: 761870273 } })
125882026-03-24T21:06:35.548ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 15.872298ms iteration = 20 status = 0
125892026-03-24T21:06:35.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
125902026-03-24T21:06:35.549ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
125912026-03-24T21:06:35.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
125922026-03-24T21:06:35.550ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1381, tv_nsec: 763869927 } })
125932026-03-24T21:06:35.554ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125942026-03-24T21:06:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125952026-03-24T21:06:35.557ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
125962026-03-24T21:06:35.557ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125972026-03-24T21:06:35.557ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
125982026-03-24T21:06:35.557ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
125992026-03-24T21:06:35.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
126002026-03-24T21:06:35.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
126012026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
126022026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
126032026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
126042026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
126052026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
126062026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
126072026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
126082026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
126092026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
126102026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
126112026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
126122026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
126132026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
126142026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
126152026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
126162026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
126172026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
126182026-03-24T21:06:35.560ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
126192026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
126202026-03-24T21:06:35.560ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
126212026-03-24T21:06:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
126222026-03-24T21:06:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
126232026-03-24T21:06:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
126242026-03-24T21:06:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
126252026-03-24T21:06:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
126262026-03-24T21:06:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
126272026-03-24T21:06:35.562ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.138038ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
126282026-03-24T21:06:35.581ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 20 reason = Timeout
126292026-03-24T21:06:35.581ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 26.141\u{b5}s iteration = 20 status = {"disabled":true,"errors":[],"sps":[]}
126302026-03-24T21:06:35.581ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 39 reason = Timeout
126312026-03-24T21:06:35.581ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
126322026-03-24T21:06:35.581ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 39 reason = Timeout
126332026-03-24T21:06:35.581ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
126342026-03-24T21:06:35.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
126352026-03-24T21:06:35.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
126362026-03-24T21:06:35.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
126372026-03-24T21:06:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
126382026-03-24T21:06:35.595ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
126392026-03-24T21:06:35.595ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
126402026-03-24T21:06:35.596ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
126412026-03-24T21:06:35.596ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
126422026-03-24T21:06:35.596ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
126432026-03-24T21:06:35.596ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
126442026-03-24T21:06:35.596ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
126452026-03-24T21:06:35.596ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
126462026-03-24T21:06:35.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
126472026-03-24T21:06:35.603ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
126482026-03-24T21:06:35.603ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
126492026-03-24T21:06:35.623ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
126502026-03-24T21:06:35.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
126512026-03-24T21:06:35.625ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126522026-03-24T21:06:35.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
126532026-03-24T21:06:35.629ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
126542026-03-24T21:06:35.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
126552026-03-24T21:06:35.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126562026-03-24T21:06:35.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
126572026-03-24T21:06:35.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
126582026-03-24T21:06:35.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
126592026-03-24T21:06:35.639ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 78.675191ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
126602026-03-24T21:06:35.640ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
126612026-03-24T21:06:35.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
126622026-03-24T21:06:35.642ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
126632026-03-24T21:06:35.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
126642026-03-24T21:06:35.644ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 86.677427ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
126652026-03-24T21:06:35.644ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 84.161509ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
126662026-03-24T21:06:35.644ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
126672026-03-24T21:06:35.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
126682026-03-24T21:06:35.645ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
126692026-03-24T21:06:35.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
126702026-03-24T21:06:35.647ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126712026-03-24T21:06:35.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
126722026-03-24T21:06:35.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
126732026-03-24T21:06:35.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
126742026-03-24T21:06:35.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126752026-03-24T21:06:35.654ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126762026-03-24T21:06:35.657ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
126772026-03-24T21:06:35.657ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
126782026-03-24T21:06:35.657ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
126792026-03-24T21:06:35.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
126802026-03-24T21:06:35.658ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
126812026-03-24T21:06:35.658ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
126822026-03-24T21:06:35.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
126832026-03-24T21:06:35.659ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
126842026-03-24T21:06:35.659ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
126852026-03-24T21:06:35.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
126862026-03-24T21:06:35.663ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126872026-03-24T21:06:35.663ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 82.112564ms iteration = 39 status = "NoSitrep"
126882026-03-24T21:06:35.663ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
126892026-03-24T21:06:35.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
126902026-03-24T21:06:35.665ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126912026-03-24T21:06:35.666ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
126922026-03-24T21:06:35.666ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
126932026-03-24T21:06:35.666ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
126942026-03-24T21:06:35.666ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 134.365693ms iteration = 58 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
126952026-03-24T21:06:35.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
126962026-03-24T21:06:35.668ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
126972026-03-24T21:06:35.671ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
126982026-03-24T21:06:35.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
126992026-03-24T21:06:35.673ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
127002026-03-24T21:06:35.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
127012026-03-24T21:06:35.697ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
127022026-03-24T21:06:35.698ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 165.368019ms iteration = 30 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
127032026-03-24T21:06:35.713ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 152.243235ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
127042026-03-24T21:06:35.715ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 168.512084ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
127052026-03-24T21:06:35.729ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
127062026-03-24T21:06:35.729ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
127072026-03-24T21:06:35.729ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
127082026-03-24T21:06:35.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
127092026-03-24T21:06:35.731ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 170.452477ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
127102026-03-24T21:06:35.732ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127112026-03-24T21:06:35.733ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
127122026-03-24T21:06:35.733ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 172.85485ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
127132026-03-24T21:06:35.733ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
127142026-03-24T21:06:35.738ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
127152026-03-24T21:06:35.738ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
127162026-03-24T21:06:35.738ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 156.87285ms iteration = 39 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
127172026-03-24T21:06:35.761ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 228.4473ms iteration = 33 status = {"physical_disks_added":0}
127182026-03-24T21:06:35.770ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
127192026-03-24T21:06:35.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
127202026-03-24T21:06:35.773ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127212026-03-24T21:06:35.775ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
127222026-03-24T21:06:35.775ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
127232026-03-24T21:06:35.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
127242026-03-24T21:06:35.775ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
127252026-03-24T21:06:35.775ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
127262026-03-24T21:06:35.777ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
127272026-03-24T21:06:35.777ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
127282026-03-24T21:06:35.777ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 216.731844ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
127292026-03-24T21:06:35.778ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "730ccfaf-091e-4f29-81fa-f5a9d3996906", "content-length": "372", "date": "Tue, 24 Mar 2026 21:06:35 GMT"} })
127302026-03-24T21:06:35.778ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127312026-03-24T21:06:35.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127322026-03-24T21:06:35.831ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127332026-03-24T21:06:35.833ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
127342026-03-24T21:06:35.833ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
127352026-03-24T21:06:35.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
127362026-03-24T21:06:35.837ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
127372026-03-24T21:06:35.838ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "38358cf2-71a7-487b-b543-6ceb3c98eabb", "content-length": "691", "date": "Tue, 24 Mar 2026 21:06:35 GMT"} })
127382026-03-24T21:06:35.838ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
127392026-03-24T21:06:35.838ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
127402026-03-24T21:06:35.838ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
127412026-03-24T21:06:35.838ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 306.217807ms iteration = 20 status = {}
127422026-03-24T21:06:35.850ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
127432026-03-24T21:06:35.850ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
127442026-03-24T21:06:35.853ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f17a5163-dc15-46f9-8b45-409b5e1831d4", "date": "Tue, 24 Mar 2026 21:06:35 GMT"} })
127452026-03-24T21:06:35.853ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127462026-03-24T21:06:35.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127472026-03-24T21:06:35.855ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127482026-03-24T21:06:35.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127492026-03-24T21:06:35.859ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
127502026-03-24T21:06:35.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
127512026-03-24T21:06:35.863ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
127522026-03-24T21:06:35.863ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
127532026-03-24T21:06:35.870ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "518135a9-248f-4ca5-8e26-4e7d277eea1a", "date": "Tue, 24 Mar 2026 21:06:35 GMT"} })
127542026-03-24T21:06:35.870ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 309.419224ms iteration = 30 status = {}
127552026-03-24T21:06:35.908ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c0467b45-8643-4de9-8faf-54080ba1bc79", "date": "Tue, 24 Mar 2026 21:06:35 GMT"} })
127562026-03-24T21:06:35.908ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
127572026-03-24T21:06:35.908ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127582026-03-24T21:06:35.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127592026-03-24T21:06:35.910ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127602026-03-24T21:06:35.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127612026-03-24T21:06:35.914ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127622026-03-24T21:06:35.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127632026-03-24T21:06:35.916ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
127642026-03-24T21:06:35.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
127652026-03-24T21:06:35.918ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 386.06381ms iteration = 20 status = {}
127662026-03-24T21:06:38.551ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 5b2e26ec-4fd6-47e1-b20b-908103686515 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
127672026-03-24T21:06:38.552ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 5b2e26ec-4fd6-47e1-b20b-908103686515 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
127682026-03-24T21:06:38.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 5b2e26ec-4fd6-47e1-b20b-908103686515 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
127692026-03-24T21:06:38.555ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 3114 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 5b2e26ec-4fd6-47e1-b20b-908103686515 response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
127702026-03-24T21:06:38.555ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 460084cd-34ea-469d-a799-0767a38b6de4 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
127712026-03-24T21:06:38.555ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 460084cd-34ea-469d-a799-0767a38b6de4 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
127722026-03-24T21:06:38.555ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 460084cd-34ea-469d-a799-0767a38b6de4 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
127732026-03-24T21:06:38.557ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 1905 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 460084cd-34ea-469d-a799-0767a38b6de4 response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
127742026-03-24T21:06:38.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 327 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 4c4ebab4-be81-4d0b-8186-efcd9988f5d0 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
127752026-03-24T21:06:40.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 116 reason = Timeout
127762026-03-24T21:06:40.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
127772026-03-24T21:06:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
127782026-03-24T21:06:40.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127792026-03-24T21:06:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
127802026-03-24T21:06:40.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
127812026-03-24T21:06:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
127822026-03-24T21:06:40.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
127832026-03-24T21:06:40.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.525453ms iteration = 116 status = {"config_updated":false}
127842026-03-24T21:06:45.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
127852026-03-24T21:06:45.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 117 reason = Timeout
127862026-03-24T21:06:45.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
127872026-03-24T21:06:45.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
127882026-03-24T21:06:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
127892026-03-24T21:06:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
127902026-03-24T21:06:45.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127912026-03-24T21:06:45.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127922026-03-24T21:06:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
127932026-03-24T21:06:45.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
127942026-03-24T21:06:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
127952026-03-24T21:06:45.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
127962026-03-24T21:06:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
127972026-03-24T21:06:45.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
127982026-03-24T21:06:45.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
127992026-03-24T21:06:45.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.883666ms iteration = 117 status = {"config_updated":false}
128002026-03-24T21:06:45.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128012026-03-24T21:06:45.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
128022026-03-24T21:06:45.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128032026-03-24T21:06:45.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
128042026-03-24T21:06:45.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128052026-03-24T21:06:45.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
128062026-03-24T21:06:45.557ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
128072026-03-24T21:06:45.557ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.344326ms iteration = 59 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
128082026-03-24T21:06:45.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
128092026-03-24T21:06:45.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
128102026-03-24T21:06:45.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
128112026-03-24T21:06:45.600ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
128122026-03-24T21:06:45.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
128132026-03-24T21:06:45.602ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.907221ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
128142026-03-24T21:06:48.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 361 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 6780fe81-5985-4e2c-95b1-85fb9dd1823d response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
128152026-03-24T21:06:50.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 52 reason = Timeout
128162026-03-24T21:06:50.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 118 reason = Timeout
128172026-03-24T21:06:50.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
128182026-03-24T21:06:50.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
128192026-03-24T21:06:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
128202026-03-24T21:06:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
128212026-03-24T21:06:50.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128222026-03-24T21:06:50.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128232026-03-24T21:06:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
128242026-03-24T21:06:50.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
128252026-03-24T21:06:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
128262026-03-24T21:06:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
128272026-03-24T21:06:50.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
128282026-03-24T21:06:50.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
128292026-03-24T21:06:50.540ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = df496995-f9d8-4235-b1db-c1cf67df3a52 original_time_started = 2026-03-24 21:06:06.720501 UTC
128302026-03-24T21:06:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.188823ms iteration = 52 status = {"Loaded":{"collection_id":"df496995-f9d8-4235-b1db-c1cf67df3a52","time_loaded":"2026-03-24T21:06:50.533090795Z","time_started":"2026-03-24T21:06:06.720501Z"}}
128312026-03-24T21:06:50.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
128322026-03-24T21:06:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.373409ms iteration = 118 status = {"config_updated":false}
128332026-03-24T21:06:50.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 40 reason = Timeout
128342026-03-24T21:06:50.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
128352026-03-24T21:06:50.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 40 reason = Timeout
128362026-03-24T21:06:50.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
128372026-03-24T21:06:50.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
128382026-03-24T21:06:50.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
128392026-03-24T21:06:50.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
128402026-03-24T21:06:50.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
128412026-03-24T21:06:50.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128422026-03-24T21:06:50.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128432026-03-24T21:06:50.586ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
128442026-03-24T21:06:50.586ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
128452026-03-24T21:06:50.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
128462026-03-24T21:06:50.586ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
128472026-03-24T21:06:50.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.970061ms iteration = 40 status = "NoSitrep"
128482026-03-24T21:06:50.589ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
128492026-03-24T21:06:50.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
128502026-03-24T21:06:50.589ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
128512026-03-24T21:06:50.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
128522026-03-24T21:06:50.591ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128532026-03-24T21:06:50.592ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
128542026-03-24T21:06:50.596ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
128552026-03-24T21:06:50.596ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
128562026-03-24T21:06:50.596ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 14.197243ms iteration = 40 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
128572026-03-24T21:06:55.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
128582026-03-24T21:06:55.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 119 reason = Timeout
128592026-03-24T21:06:55.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
128602026-03-24T21:06:55.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128612026-03-24T21:06:55.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
128622026-03-24T21:06:55.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
128632026-03-24T21:06:55.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128642026-03-24T21:06:55.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128652026-03-24T21:06:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
128662026-03-24T21:06:55.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128672026-03-24T21:06:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
128682026-03-24T21:06:55.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
128692026-03-24T21:06:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
128702026-03-24T21:06:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
128712026-03-24T21:06:55.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
128722026-03-24T21:06:55.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.203264ms iteration = 119 status = {"config_updated":false}
128732026-03-24T21:06:55.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128742026-03-24T21:06:55.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
128752026-03-24T21:06:55.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128762026-03-24T21:06:55.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
128772026-03-24T21:06:55.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
128782026-03-24T21:06:55.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
128792026-03-24T21:06:55.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
128802026-03-24T21:06:55.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
128812026-03-24T21:06:55.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
128822026-03-24T21:06:55.564ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
128832026-03-24T21:06:55.564ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.476942ms iteration = 60 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
128842026-03-24T21:06:55.602ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
128852026-03-24T21:06:55.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
128862026-03-24T21:06:55.649ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 88.446785ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
128872026-03-24T21:06:58.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 398 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = ddc93f15-5e1c-49b9-87ca-47d807815395 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
128882026-03-24T21:07:00.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 120 reason = Timeout
128892026-03-24T21:07:00.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
128902026-03-24T21:07:00.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
128912026-03-24T21:07:00.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128922026-03-24T21:07:00.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
128932026-03-24T21:07:00.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
128942026-03-24T21:07:00.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
128952026-03-24T21:07:00.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
128962026-03-24T21:07:00.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.332718ms iteration = 120 status = {"config_updated":false}
128972026-03-24T21:07:05.516ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
128982026-03-24T21:07:05.516ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
128992026-03-24T21:07:05.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
129002026-03-24T21:07:05.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
129012026-03-24T21:07:05.517ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
129022026-03-24T21:07:05.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
129032026-03-24T21:07:05.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129042026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
129052026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
129062026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
129072026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 21 reason = Timeout
129082026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
129092026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
129102026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
129112026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
129122026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
129132026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
129142026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
129152026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
129162026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
129172026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
129182026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
129192026-03-24T21:07:05.521ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2026-03-24 20:57:05.521176897 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
129202026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
129212026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
129222026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
129232026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
129242026-03-24T21:07:05.521ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
129252026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
129262026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
129272026-03-24T21:07:05.521ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = ExternalNetworking step id = Ensure
129282026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
129292026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
129302026-03-24T21:07:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
129312026-03-24T21:07:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("27f3a9c5-67c7-41c5-994d-9db513cda557.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2f7ea2ed-fd45-4cd7-9fb1-f48119541118.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 735606206 } })
129322026-03-24T21:07:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("93b6e989-1f7d-4884-aec9-ecbc076ed6c4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 735625386 } })
129332026-03-24T21:07:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
129342026-03-24T21:07:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
129352026-03-24T21:07:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.501431ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
129362026-03-24T21:07:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
129372026-03-24T21:07:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
129382026-03-24T21:07:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.513871ms iteration = 11 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
129392026-03-24T21:07:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
129402026-03-24T21:07:05.523ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
129412026-03-24T21:07:05.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
129422026-03-24T21:07:05.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
129432026-03-24T21:07:05.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
129442026-03-24T21:07:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "1dc77494-5f3f-466c-a344-f67afc4bb9c9", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
129452026-03-24T21:07:05.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
129462026-03-24T21:07:05.524ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2026-03-24 20:57:05.521176897 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
129472026-03-24T21:07:05.524ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.674269ms iteration = 11 status = {"expiration":"2026-03-24T20:57:05.521176897Z","pruned":[]}
129482026-03-24T21:07:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "00180975-092b-4302-a9b8-426438bc436a", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
129492026-03-24T21:07:05.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
129502026-03-24T21:07:05.525ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
129512026-03-24T21:07:05.525ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 3.965837ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
129522026-03-24T21:07:05.525ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 739077109 } })
129532026-03-24T21:07:05.525ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
129542026-03-24T21:07:05.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
129552026-03-24T21:07:05.526ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 739871261 } })
129562026-03-24T21:07:05.526ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
129572026-03-24T21:07:05.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
129582026-03-24T21:07:05.526ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129592026-03-24T21:07:05.526ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129602026-03-24T21:07:05.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
129612026-03-24T21:07:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "73e7cb3f-3528-48c4-b1b2-5bb4860c5591", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
129622026-03-24T21:07:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
129632026-03-24T21:07:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d9e74997-5fcf-4262-abbd-ea08f171baf6", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
129642026-03-24T21:07:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
129652026-03-24T21:07:05.527ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "a94c14ef-67a8-4593-814b-5510c2d8eedb", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
129662026-03-24T21:07:05.527ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
129672026-03-24T21:07:05.527ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
129682026-03-24T21:07:05.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 5.799186ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
129692026-03-24T21:07:05.527ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 740838147 } })
129702026-03-24T21:07:05.527ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.489205ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
129712026-03-24T21:07:05.528ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
129722026-03-24T21:07:05.528ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 7.0417ms iteration = 21 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
129732026-03-24T21:07:05.528ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
129742026-03-24T21:07:05.528ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
129752026-03-24T21:07:05.529ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "76d9f494-e6b5-47d3-b35c-22a47ff03662", "content-length": "26", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} })
129762026-03-24T21:07:05.529ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
129772026-03-24T21:07:05.529ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
129782026-03-24T21:07:05.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
129792026-03-24T21:07:05.529ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 743573991 } })
129802026-03-24T21:07:05.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
129812026-03-24T21:07:05.530ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
129822026-03-24T21:07:05.530ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
129832026-03-24T21:07:05.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
129842026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
129852026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 16 reason = Timeout
129862026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 11.326536ms iteration = 21 status = {}
129872026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Timeout
129882026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
129892026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 26 reason = Timeout
129902026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 31 reason = Timeout
129912026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
129922026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129932026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
129942026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
129952026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
129962026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 53 reason = Timeout
129972026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
129982026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
129992026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130002026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 121 reason = Timeout
130012026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
130022026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
130032026-03-24T21:07:05.532ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
130042026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
130052026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
130062026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
130072026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
130082026-03-24T21:07:05.532ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130092026-03-24T21:07:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
130102026-03-24T21:07:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
130112026-03-24T21:07:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
130122026-03-24T21:07:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
130132026-03-24T21:07:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
130142026-03-24T21:07:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
130152026-03-24T21:07:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130162026-03-24T21:07:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
130172026-03-24T21:07:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130182026-03-24T21:07:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
130192026-03-24T21:07:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
130202026-03-24T21:07:05.535ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130212026-03-24T21:07:05.536ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130222026-03-24T21:07:05.536ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130232026-03-24T21:07:05.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130242026-03-24T21:07:05.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
130252026-03-24T21:07:05.540ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
130262026-03-24T21:07:05.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
130272026-03-24T21:07:05.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130282026-03-24T21:07:05.541ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
130292026-03-24T21:07:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
130302026-03-24T21:07:05.541ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
130312026-03-24T21:07:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
130322026-03-24T21:07:05.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
130332026-03-24T21:07:05.541ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
130342026-03-24T21:07:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
130352026-03-24T21:07:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
130362026-03-24T21:07:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
130372026-03-24T21:07:05.544ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
130382026-03-24T21:07:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
130392026-03-24T21:07:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
130402026-03-24T21:07:05.546ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
130412026-03-24T21:07:05.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
130422026-03-24T21:07:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
130432026-03-24T21:07:05.550ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
130442026-03-24T21:07:05.551ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 17.969935ms iteration = 121 status = {"config_updated":false}
130452026-03-24T21:07:05.552ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
130462026-03-24T21:07:05.555ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
130472026-03-24T21:07:05.555ZINFOnexus (SagaQuiesceHandle): saga recovery pass starting file = nexus/types/src/quiesce.rs:510
130482026-03-24T21:07:05.555ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
130492026-03-24T21:07:05.555ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
130502026-03-24T21:07:05.555ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
130512026-03-24T21:07:05.555ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
130522026-03-24T21:07:05.555ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
130532026-03-24T21:07:05.555ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
130542026-03-24T21:07:05.555ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
130552026-03-24T21:07:05.555ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
130562026-03-24T21:07:05.555ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
130572026-03-24T21:07:05.555ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
130582026-03-24T21:07:05.556ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
130592026-03-24T21:07:05.556ZDEBGnexus (BackgroundTasks): activating background_task = tuf_repo_pruner iteration = 4 reason = Timeout
130602026-03-24T21:07:05.556ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
130612026-03-24T21:07:05.556ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a (built_in_user), .. }) resource = Database result = Ok(())
130622026-03-24T21:07:05.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
130632026-03-24T21:07:05.556ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 770223719 } })
130642026-03-24T21:07:05.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
130652026-03-24T21:07:05.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
130662026-03-24T21:07:05.556ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
130672026-03-24T21:07:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
130682026-03-24T21:07:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
130692026-03-24T21:07:05.557ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 24 reason = Timeout
130702026-03-24T21:07:05.557ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
130712026-03-24T21:07:05.557ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 11 reason = Timeout
130722026-03-24T21:07:05.557ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 20.33\u{b5}s iteration = 24 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
130732026-03-24T21:07:05.557ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
130742026-03-24T21:07:05.557ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
130752026-03-24T21:07:05.557ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 11 reason = Timeout
130762026-03-24T21:07:05.557ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
130772026-03-24T21:07:05.557ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
130782026-03-24T21:07:05.557ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 771256517 } })
130792026-03-24T21:07:05.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
130802026-03-24T21:07:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
130812026-03-24T21:07:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
130822026-03-24T21:07:05.559ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
130832026-03-24T21:07:05.559ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
130842026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
130852026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
130862026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
130872026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
130882026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
130892026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
130902026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
130912026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
130922026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
130932026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
130942026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
130952026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
130962026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
130972026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
130982026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
130992026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
131002026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
131012026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
131022026-03-24T21:07:05.560ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
131032026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
131042026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
131052026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131062026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131072026-03-24T21:07:05.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131082026-03-24T21:07:05.561ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131092026-03-24T21:07:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
131102026-03-24T21:07:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
131112026-03-24T21:07:05.561ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "368f10de-6f6f-4df8-95e6-d26d95248a71", "content-length": "26", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} })
131122026-03-24T21:07:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
131132026-03-24T21:07:05.561ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
131142026-03-24T21:07:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
131152026-03-24T21:07:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
131162026-03-24T21:07:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
131172026-03-24T21:07:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
131182026-03-24T21:07:05.561ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
131192026-03-24T21:07:05.562ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131202026-03-24T21:07:05.562ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.392362ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
131212026-03-24T21:07:05.563ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131222026-03-24T21:07:05.563ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 776972801 } })
131232026-03-24T21:07:05.563ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
131242026-03-24T21:07:05.563ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131252026-03-24T21:07:05.564ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
131262026-03-24T21:07:05.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
131272026-03-24T21:07:05.564ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
131282026-03-24T21:07:05.564ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
131292026-03-24T21:07:05.564ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 7.439391ms iteration = 11 status = {"by_rx":{},"error":null}
131302026-03-24T21:07:05.566ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 779946361 } })
131312026-03-24T21:07:05.566ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
131322026-03-24T21:07:05.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
131332026-03-24T21:07:05.566ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
131342026-03-24T21:07:05.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
131352026-03-24T21:07:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
131362026-03-24T21:07:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
131372026-03-24T21:07:05.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131382026-03-24T21:07:05.567ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
131392026-03-24T21:07:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131402026-03-24T21:07:05.568ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
131412026-03-24T21:07:05.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
131422026-03-24T21:07:05.568ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
131432026-03-24T21:07:05.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
131442026-03-24T21:07:05.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
131452026-03-24T21:07:05.569ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
131462026-03-24T21:07:05.569ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131472026-03-24T21:07:05.571ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
131482026-03-24T21:07:05.571ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
131492026-03-24T21:07:05.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
131502026-03-24T21:07:05.573ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
131512026-03-24T21:07:05.573ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
131522026-03-24T21:07:05.573ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
131532026-03-24T21:07:05.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
131542026-03-24T21:07:05.574ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
131552026-03-24T21:07:05.574ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
131562026-03-24T21:07:05.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
131572026-03-24T21:07:05.576ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
131582026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): activating background_task = audit_log_cleanup iteration = 2 reason = Timeout
131592026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): roles background_task = audit_log_cleanup roles = RoleSet { roles: {} }
131602026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): activating background_task = trust_quorum_manager iteration = 11 reason = Timeout
131612026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): activating background_task = fm_rendezvous iteration = 3 reason = Timeout
131622026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_rendezvous elapsed = 9.081\u{b5}s iteration = 3 status = "NoSitrep"
131632026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): activating background_task = audit_log_timeout_incomplete iteration = 2 reason = Timeout
131642026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
131652026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 11 reason = Timeout
131662026-03-24T21:07:05.576ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
131672026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): roles background_task = audit_log_timeout_incomplete roles = RoleSet { roles: {} }
131682026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): activating background_task = session_cleanup iteration = 3 reason = Timeout
131692026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
131702026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): activating background_task = attached_subnet_manager iteration = 11 reason = Timeout
131712026-03-24T21:07:05.576ZDEBGnexus (BackgroundTasks): roles background_task = session_cleanup roles = RoleSet { roles: {} }
131722026-03-24T21:07:05.576ZINFOnexus (BackgroundTasks): starting attached subnet manager work background_task = attached_subnet_manager file = nexus/src/app/background/tasks/attached_subnets.rs:374
131732026-03-24T21:07:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = session_cleanup resource = Database result = Ok(())
131742026-03-24T21:07:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
131752026-03-24T21:07:05.577ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
131762026-03-24T21:07:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = audit_log_timeout_incomplete resource = Database result = Ok(())
131772026-03-24T21:07:05.577ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1411, tv_nsec: 790906907 } })
131782026-03-24T21:07:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
131792026-03-24T21:07:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
131802026-03-24T21:07:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = audit_log_cleanup resource = Database result = Ok(())
131812026-03-24T21:07:05.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
131822026-03-24T21:07:05.579ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
131832026-03-24T21:07:05.580ZDEBGnexus (BackgroundTasks): roles background_task = attached_subnet_manager roles = RoleSet { roles: {} }
131842026-03-24T21:07:05.580ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
131852026-03-24T21:07:05.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = attached_subnet_manager resource = Database result = Ok(())
131862026-03-24T21:07:05.582ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 21 reason = Timeout
131872026-03-24T21:07:05.582ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.98\u{b5}s iteration = 21 status = {"disabled":true,"errors":[],"sps":[]}
131882026-03-24T21:07:05.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 41 reason = Timeout
131892026-03-24T21:07:05.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
131902026-03-24T21:07:05.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 41 reason = Timeout
131912026-03-24T21:07:05.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
131922026-03-24T21:07:05.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
131932026-03-24T21:07:05.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
131942026-03-24T21:07:05.584ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
131952026-03-24T21:07:05.585ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
131962026-03-24T21:07:05.590ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
131972026-03-24T21:07:05.590ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = SledList step id = Fetch
131982026-03-24T21:07:05.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
131992026-03-24T21:07:05.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132002026-03-24T21:07:05.657ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132012026-03-24T21:07:05.660ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
132022026-03-24T21:07:05.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132032026-03-24T21:07:05.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132042026-03-24T21:07:05.661ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
132052026-03-24T21:07:05.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
132062026-03-24T21:07:05.663ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132072026-03-24T21:07:05.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132082026-03-24T21:07:05.666ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
132092026-03-24T21:07:05.666ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
132102026-03-24T21:07:05.666ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 134.134477ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
132112026-03-24T21:07:05.667ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 107.455008ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
132122026-03-24T21:07:05.668ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
132132026-03-24T21:07:05.668ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132142026-03-24T21:07:05.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132152026-03-24T21:07:05.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
132162026-03-24T21:07:05.669ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
132172026-03-24T21:07:05.670ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
132182026-03-24T21:07:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
132192026-03-24T21:07:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
132202026-03-24T21:07:05.672ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132212026-03-24T21:07:05.672ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
132222026-03-24T21:07:05.672ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
132232026-03-24T21:07:05.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132242026-03-24T21:07:05.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132252026-03-24T21:07:05.672ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:473
132262026-03-24T21:07:05.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
132272026-03-24T21:07:05.672ZINFOnexus (SagaQuiesceHandle): saga recovery pass finished blueprint_id = Some(9e1c09c1-3960-4ecf-a904-19e137699ab3 (blueprint)) file = nexus/types/src/quiesce.rs:525 generation = 1
132282026-03-24T21:07:05.673ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 117.456097ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
132292026-03-24T21:07:05.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
132302026-03-24T21:07:05.675ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
132312026-03-24T21:07:05.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
132322026-03-24T21:07:05.676ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132332026-03-24T21:07:05.677ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
132342026-03-24T21:07:05.677ZDEBGnexus (BackgroundTasks): roles background_task = session_cleanup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132352026-03-24T21:07:05.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
132362026-03-24T21:07:05.677ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
132372026-03-24T21:07:05.678ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
132382026-03-24T21:07:05.678ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
132392026-03-24T21:07:05.678ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
132402026-03-24T21:07:05.678ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
132412026-03-24T21:07:05.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
132422026-03-24T21:07:05.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
132432026-03-24T21:07:05.679ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132442026-03-24T21:07:05.680ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = session_cleanup resource = Fleet result = Ok(())
132452026-03-24T21:07:05.680ZDEBGnexus (BackgroundTasks): roles background_task = session_cleanup roles = RoleSet { roles: {} }
132462026-03-24T21:07:05.680ZDEBGnexus (BackgroundTasks): roles background_task = audit_log_cleanup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132472026-03-24T21:07:05.680ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132482026-03-24T21:07:05.680ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
132492026-03-24T21:07:05.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = session_cleanup resource = Database result = Ok(())
132502026-03-24T21:07:05.681ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
132512026-03-24T21:07:05.681ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
132522026-03-24T21:07:05.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
132532026-03-24T21:07:05.681ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
132542026-03-24T21:07:05.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
132552026-03-24T21:07:05.681ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
132562026-03-24T21:07:05.681ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
132572026-03-24T21:07:05.681ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
132582026-03-24T21:07:05.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
132592026-03-24T21:07:05.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
132602026-03-24T21:07:05.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132612026-03-24T21:07:05.682ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132622026-03-24T21:07:05.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132632026-03-24T21:07:05.683ZDEBGnexus (BackgroundTasks): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = audit_log_cleanup resource = AuditLog result = Ok(())
132642026-03-24T21:07:05.683ZDEBGnexus (BackgroundTasks): roles background_task = audit_log_cleanup roles = RoleSet { roles: {} }
132652026-03-24T21:07:05.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = audit_log_cleanup resource = Database result = Ok(())
132662026-03-24T21:07:05.684ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
132672026-03-24T21:07:05.684ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
132682026-03-24T21:07:05.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
132692026-03-24T21:07:05.685ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 152.583274ms iteration = 21 status = 0
132702026-03-24T21:07:05.685ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Fleet result = Ok(())
132712026-03-24T21:07:05.685ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
132722026-03-24T21:07:05.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
132732026-03-24T21:07:05.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
132742026-03-24T21:07:05.686ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
132752026-03-24T21:07:05.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
132762026-03-24T21:07:05.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
132772026-03-24T21:07:05.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
132782026-03-24T21:07:05.687ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
132792026-03-24T21:07:05.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
132802026-03-24T21:07:05.687ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
132812026-03-24T21:07:05.687ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
132822026-03-24T21:07:05.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
132832026-03-24T21:07:05.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
132842026-03-24T21:07:05.689ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
132852026-03-24T21:07:05.690ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
132862026-03-24T21:07:05.690ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 134.080415ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
132872026-03-24T21:07:05.693ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132882026-03-24T21:07:05.693ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132892026-03-24T21:07:05.693ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
132902026-03-24T21:07:05.693ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 136.362967ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
132912026-03-24T21:07:05.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
132922026-03-24T21:07:05.694ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
132932026-03-24T21:07:05.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
132942026-03-24T21:07:05.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132952026-03-24T21:07:05.697ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 139.886072ms iteration = 11 status = {"dispatched":[],"errors":[],"glob_version":"242.0.0","globs_reprocessed":{},"no_receivers":[]}
132962026-03-24T21:07:05.698ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
132972026-03-24T21:07:05.698ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
132982026-03-24T21:07:05.698ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
132992026-03-24T21:07:05.698ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
133002026-03-24T21:07:05.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
133012026-03-24T21:07:05.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133022026-03-24T21:07:05.698ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133032026-03-24T21:07:05.702ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
133042026-03-24T21:07:05.702ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
133052026-03-24T21:07:05.702ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133062026-03-24T21:07:05.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
133072026-03-24T21:07:05.702ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
133082026-03-24T21:07:05.702ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
133092026-03-24T21:07:05.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
133102026-03-24T21:07:05.704ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133112026-03-24T21:07:05.707ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
133122026-03-24T21:07:05.707ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 147.197209ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
133132026-03-24T21:07:05.708ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
133142026-03-24T21:07:05.708ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
133152026-03-24T21:07:05.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
133162026-03-24T21:07:05.709ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
133172026-03-24T21:07:05.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
133182026-03-24T21:07:05.711ZDEBGnexus (BackgroundTasks): session cleanup found no expired sessions background_task = session_cleanup cutoff = 2026-03-23 21:07:05.576471628 UTC
133192026-03-24T21:07:05.711ZDEBGnexus (BackgroundTasks): activation complete background_task = session_cleanup elapsed = 134.802885ms iteration = 3 status = {"cutoff":"2026-03-23T21:07:05.576471628Z","deleted":0,"error":null,"limit":10000}
133202026-03-24T21:07:05.711ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
133212026-03-24T21:07:05.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
133222026-03-24T21:07:05.713ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133232026-03-24T21:07:05.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133242026-03-24T21:07:05.718ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
133252026-03-24T21:07:05.718ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
133262026-03-24T21:07:05.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
133272026-03-24T21:07:05.719ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
133282026-03-24T21:07:05.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
133292026-03-24T21:07:05.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
133302026-03-24T21:07:05.720ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133312026-03-24T21:07:05.720ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 160.2113ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
133322026-03-24T21:07:05.720ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
133332026-03-24T21:07:05.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
133342026-03-24T21:07:05.723ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 141.68418ms iteration = 41 status = "NoSitrep"
133352026-03-24T21:07:05.724ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
133362026-03-24T21:07:05.724ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133372026-03-24T21:07:05.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
133382026-03-24T21:07:05.724ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
133392026-03-24T21:07:05.724ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
133402026-03-24T21:07:05.724ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
133412026-03-24T21:07:05.724ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
133422026-03-24T21:07:05.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
133432026-03-24T21:07:05.726ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
133442026-03-24T21:07:05.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133452026-03-24T21:07:05.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133462026-03-24T21:07:05.726ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133472026-03-24T21:07:05.729ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
133482026-03-24T21:07:05.729ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
133492026-03-24T21:07:05.729ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133502026-03-24T21:07:05.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
133512026-03-24T21:07:05.731ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
133522026-03-24T21:07:05.731ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133532026-03-24T21:07:05.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133542026-03-24T21:07:05.732ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 185.958145ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
133552026-03-24T21:07:05.733ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
133562026-03-24T21:07:05.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
133572026-03-24T21:07:05.735ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133582026-03-24T21:07:05.736ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
133592026-03-24T21:07:05.736ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
133602026-03-24T21:07:05.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
133612026-03-24T21:07:05.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133622026-03-24T21:07:05.739ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
133632026-03-24T21:07:05.739ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
133642026-03-24T21:07:05.739ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
133652026-03-24T21:07:05.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
133662026-03-24T21:07:05.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
133672026-03-24T21:07:05.740ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
133682026-03-24T21:07:05.740ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
133692026-03-24T21:07:05.741ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
133702026-03-24T21:07:05.741ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
133712026-03-24T21:07:05.741ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
133722026-03-24T21:07:05.741ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
133732026-03-24T21:07:05.741ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
133742026-03-24T21:07:05.741ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133752026-03-24T21:07:05.741ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
133762026-03-24T21:07:05.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
133772026-03-24T21:07:05.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133782026-03-24T21:07:05.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
133792026-03-24T21:07:05.783ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
133802026-03-24T21:07:05.783ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 251.383729ms iteration = 61 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
133812026-03-24T21:07:05.785ZDEBGnexus (BackgroundTasks): roles background_task = audit_log_timeout_incomplete roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133822026-03-24T21:07:05.787ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = DeployNexusRecords step id = Ensure
133832026-03-24T21:07:05.787ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = df496995-f9d8-4235-b1db-c1cf67df3a52 original_time_started = 2026-03-24 21:06:06.720501 UTC
133842026-03-24T21:07:05.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
133852026-03-24T21:07:05.787ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 255.219542ms iteration = 53 status = {"Loaded":{"collection_id":"df496995-f9d8-4235-b1db-c1cf67df3a52","time_loaded":"2026-03-24T21:07:05.532481432Z","time_started":"2026-03-24T21:06:06.720501Z"}}
133862026-03-24T21:07:05.787ZDEBGnexus (BackgroundTasks): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = audit_log_timeout_incomplete resource = AuditLog result = Ok(())
133872026-03-24T21:07:05.788ZDEBGnexus (BackgroundTasks): roles background_task = audit_log_timeout_incomplete roles = RoleSet { roles: {} }
133882026-03-24T21:07:05.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
133892026-03-24T21:07:05.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = audit_log_timeout_incomplete resource = Database result = Ok(())
133902026-03-24T21:07:05.790ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 230.017053ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
133912026-03-24T21:07:05.790ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133922026-03-24T21:07:05.793ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133932026-03-24T21:07:05.795ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
133942026-03-24T21:07:05.795ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = TargetReleaseConfig result = Ok(())
133952026-03-24T21:07:05.795ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
133962026-03-24T21:07:05.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
133972026-03-24T21:07:05.797ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
133982026-03-24T21:07:05.797ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
133992026-03-24T21:07:05.803ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
134002026-03-24T21:07:05.804ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 243.413554ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
134012026-03-24T21:07:05.806ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
134022026-03-24T21:07:05.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
134032026-03-24T21:07:05.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
134042026-03-24T21:07:05.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134052026-03-24T21:07:05.809ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
134062026-03-24T21:07:05.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
134072026-03-24T21:07:05.811ZINFOnexus (BackgroundTasks): tuf_repo_prune decision background_task = tuf_repo_pruner file = nexus/src/app/background/tasks/tuf_repo_pruner.rs:153 status = TufRepoPrunerDetails { nkeep_recent_releases: 3, nkeep_recent_uploads: 2, repos_keep_target_release: {}, repos_keep_recent_uploads: {}, repo_prune: None, other_repos_eligible_to_prune: {}, warnings: [] }
134082026-03-24T21:07:05.811ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_repo_pruner elapsed = 255.684414ms iteration = 4 status = {"nkeep_recent_releases":3,"nkeep_recent_uploads":2,"other_repos_eligible_to_prune":[],"repo_prune":null,"repos_keep_recent_uploads":[],"repos_keep_target_release":[],"status":"enabled","warnings":[]}
134092026-03-24T21:07:05.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134102026-03-24T21:07:05.812ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
134112026-03-24T21:07:05.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134122026-03-24T21:07:05.812ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
134132026-03-24T21:07:05.812ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
134142026-03-24T21:07:05.812ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 230.632199ms iteration = 41 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
134152026-03-24T21:07:05.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134162026-03-24T21:07:05.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
134172026-03-24T21:07:05.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
134182026-03-24T21:07:05.815ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134192026-03-24T21:07:05.816ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
134202026-03-24T21:07:05.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
134212026-03-24T21:07:05.816ZDEBGnexus (BackgroundTasks): audit log cleanup: no old entries to delete background_task = audit_log_cleanup cutoff = 2025-12-24 21:07:05.576188381 UTC
134222026-03-24T21:07:05.816ZDEBGnexus (BackgroundTasks): activation complete background_task = audit_log_cleanup elapsed = 240.110994ms iteration = 2 status = {"cutoff":"2025-12-24T21:07:05.576188381Z","error":null,"max_deleted_per_activation":10000,"rows_deleted":0}
134232026-03-24T21:07:05.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
134242026-03-24T21:07:05.817ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134252026-03-24T21:07:05.818ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 257.892084ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
134262026-03-24T21:07:05.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134272026-03-24T21:07:05.819ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
134282026-03-24T21:07:05.819ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
134292026-03-24T21:07:05.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
134302026-03-24T21:07:05.823ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
134312026-03-24T21:07:05.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
134322026-03-24T21:07:05.823ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
134332026-03-24T21:07:05.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
134342026-03-24T21:07:05.832ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
134352026-03-24T21:07:05.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
134362026-03-24T21:07:05.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
134372026-03-24T21:07:05.833ZINFOnexus (BackgroundTasks): Loaded 0 active trust quorum configurations from database background_task = trust_quorum_manager bgtask = TrustQuorumManager file = nexus/src/app/background/tasks/trust_quorum.rs:83
134382026-03-24T21:07:05.833ZDEBGnexus (BackgroundTasks): activation complete background_task = trust_quorum_manager elapsed = 257.180085ms iteration = 11 status = {"PerRackStatus":{"errors":[],"statuses":[]}}
134392026-03-24T21:07:05.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134402026-03-24T21:07:05.839ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 306.596787ms iteration = 34 status = {"physical_disks_added":0}
134412026-03-24T21:07:05.840ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
134422026-03-24T21:07:05.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
134432026-03-24T21:07:05.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
134442026-03-24T21:07:05.845ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
134452026-03-24T21:07:05.846ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
134462026-03-24T21:07:05.846ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
134472026-03-24T21:07:05.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
134482026-03-24T21:07:05.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134492026-03-24T21:07:05.850ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134502026-03-24T21:07:05.850ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
134512026-03-24T21:07:05.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
134522026-03-24T21:07:05.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
134532026-03-24T21:07:05.854ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
134542026-03-24T21:07:05.854ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 321.874049ms iteration = 16 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"df496995-f9d8-4235-b1db-c1cf67df3a52","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
134552026-03-24T21:07:05.856ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
134562026-03-24T21:07:05.856ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
134572026-03-24T21:07:05.856ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
134582026-03-24T21:07:05.856ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
134592026-03-24T21:07:05.856ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
134602026-03-24T21:07:05.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
134612026-03-24T21:07:05.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
134622026-03-24T21:07:05.868ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
134632026-03-24T21:07:05.868ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
134642026-03-24T21:07:05.868ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2026-03-24 20:57:05.047625 UTC dns_group = internal generation = 1
134652026-03-24T21:07:05.868ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 352.806233ms iteration = 11 status = {"generation":1}
134662026-03-24T21:07:05.869ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
134672026-03-24T21:07:05.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
134682026-03-24T21:07:05.874ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134692026-03-24T21:07:05.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
134702026-03-24T21:07:05.881ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
134712026-03-24T21:07:05.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
134722026-03-24T21:07:05.884ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
134732026-03-24T21:07:05.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
134742026-03-24T21:07:05.887ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
134752026-03-24T21:07:05.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
134762026-03-24T21:07:05.893ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134772026-03-24T21:07:05.901ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
134782026-03-24T21:07:05.901ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 369.219296ms iteration = 31 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
134792026-03-24T21:07:05.902ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
134802026-03-24T21:07:05.902ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
134812026-03-24T21:07:05.902ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
134822026-03-24T21:07:05.902ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
134832026-03-24T21:07:05.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
134842026-03-24T21:07:05.902ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
134852026-03-24T21:07:05.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
134862026-03-24T21:07:05.905ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
134872026-03-24T21:07:05.905ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134882026-03-24T21:07:05.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
134892026-03-24T21:07:05.907ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
134902026-03-24T21:07:05.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
134912026-03-24T21:07:05.908ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134922026-03-24T21:07:05.911ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
134932026-03-24T21:07:05.911ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
134942026-03-24T21:07:05.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
134952026-03-24T21:07:05.912ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
134962026-03-24T21:07:05.912ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
134972026-03-24T21:07:05.912ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
134982026-03-24T21:07:05.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
134992026-03-24T21:07:05.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135002026-03-24T21:07:05.913ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
135012026-03-24T21:07:05.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
135022026-03-24T21:07:05.914ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
135032026-03-24T21:07:05.914ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
135042026-03-24T21:07:05.914ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2026-03-24 20:57:07.125641 UTC dns_group = external generation = 2
135052026-03-24T21:07:05.914ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 393.574472ms iteration = 11 status = {"generation":2}
135062026-03-24T21:07:05.915ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
135072026-03-24T21:07:05.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135082026-03-24T21:07:05.917ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
135092026-03-24T21:07:05.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
135102026-03-24T21:07:05.918ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
135112026-03-24T21:07:05.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135122026-03-24T21:07:05.919ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135132026-03-24T21:07:05.920ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
135142026-03-24T21:07:05.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135152026-03-24T21:07:05.921ZDEBGnexus (BackgroundTasks): listed attached subnets background_task = attached_subnet_manager n_subnets = 0
135162026-03-24T21:07:05.921ZDEBGnexus (BackgroundTasks): activation complete background_task = attached_subnet_manager elapsed = 345.094865ms iteration = 11 status = {"db_error":null,"dendrite":{"switch0":{"errors":[],"n_subnets_added":0,"n_subnets_removed":0,"n_total_subnets":0}},"sled":{}}
135172026-03-24T21:07:05.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
135182026-03-24T21:07:05.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
135192026-03-24T21:07:05.922ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
135202026-03-24T21:07:05.922ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
135212026-03-24T21:07:05.923ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
135222026-03-24T21:07:05.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135232026-03-24T21:07:05.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
135242026-03-24T21:07:05.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135252026-03-24T21:07:05.924ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135262026-03-24T21:07:05.925ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
135272026-03-24T21:07:05.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
135282026-03-24T21:07:05.927ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135292026-03-24T21:07:05.928ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2026-03-24T20:56:59.957750Z, time_modified: 2026-03-24T20:56:59.957750Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }) }
135302026-03-24T21:07:05.928ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 407.345324ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]}},"default_endpoint":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
135312026-03-24T21:07:05.928ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
135322026-03-24T21:07:05.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
135332026-03-24T21:07:05.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
135342026-03-24T21:07:05.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135352026-03-24T21:07:05.931ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
135362026-03-24T21:07:05.931ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
135372026-03-24T21:07:05.931ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
135382026-03-24T21:07:05.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
135392026-03-24T21:07:05.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
135402026-03-24T21:07:05.933ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
135412026-03-24T21:07:05.933ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = SledAgent step id = Ensure
135422026-03-24T21:07:05.933ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
135432026-03-24T21:07:05.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
135442026-03-24T21:07:05.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
135452026-03-24T21:07:05.934ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "8878a93e-aaf6-42f2-b9b1-9d62b68bd5df", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} })
135462026-03-24T21:07:05.934ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
135472026-03-24T21:07:05.934ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d4d2cd8-524c-45c6-ab66-0ca7a077c749", "content-length": "26", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} })
135482026-03-24T21:07:05.934ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:680 sled = http://[fd00:1122:3344:101::1]:12345
135492026-03-24T21:07:05.934ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
135502026-03-24T21:07:05.934ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
135512026-03-24T21:07:05.934ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 378.568488ms iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":3,"put_config_err":0,"put_config_ok":3,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"b4293d4f-3308-4dfa-b261-a103215913e4","time":"2026-03-24T20:57:06.273272074Z"},{"operation":"list","target_sled":"b4293d4f-3308-4dfa-b261-a103215913e4","time":"2026-03-24T20:57:06.274254260Z"},{"generation":1,"operation":"put_config","target_sled":"b4293d4f-3308-4dfa-b261-a103215913e4","time":"2026-03-24T21:02:06.022637029Z"},{"operation":"list","target_sled":"b4293d4f-3308-4dfa-b261-a103215913e4","time":"2026-03-24T21:02:06.023033189Z"},{"generation":1,"operation":"put_config","target_sled":"b4293d4f-3308-4dfa-b261-a103215913e4","time":"2026-03-24T21:07:05.934056181Z"},{"operation":"list","target_sled":"b4293d4f-3308-4dfa-b261-a103215913e4","time":"2026-03-24T21:07:05.934391040Z"}]}
135522026-03-24T21:07:05.934ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
135532026-03-24T21:07:05.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135542026-03-24T21:07:05.934ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
135552026-03-24T21:07:05.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
135562026-03-24T21:07:05.936ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
135572026-03-24T21:07:05.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135582026-03-24T21:07:05.937ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135592026-03-24T21:07:05.942ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
135602026-03-24T21:07:05.942ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
135612026-03-24T21:07:05.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
135622026-03-24T21:07:05.944ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
135632026-03-24T21:07:05.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135642026-03-24T21:07:05.948ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
135652026-03-24T21:07:05.948ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
135662026-03-24T21:07:05.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
135672026-03-24T21:07:05.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135682026-03-24T21:07:05.950ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135692026-03-24T21:07:05.951ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
135702026-03-24T21:07:05.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135712026-03-24T21:07:05.953ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
135722026-03-24T21:07:05.953ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
135732026-03-24T21:07:05.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
135742026-03-24T21:07:05.954ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
135752026-03-24T21:07:05.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135762026-03-24T21:07:05.956ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135772026-03-24T21:07:05.957ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "e16216af-452c-4def-8ea5-ef1b2589b2d6", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} }) sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
135782026-03-24T21:07:05.957ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = FirewallRules step id = Ensure
135792026-03-24T21:07:05.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135802026-03-24T21:07:05.957ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
135812026-03-24T21:07:05.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135822026-03-24T21:07:05.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
135832026-03-24T21:07:05.958ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
135842026-03-24T21:07:05.958ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
135852026-03-24T21:07:05.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
135862026-03-24T21:07:05.958ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
135872026-03-24T21:07:05.958ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
135882026-03-24T21:07:05.958ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 398.31202ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
135892026-03-24T21:07:05.959ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
135902026-03-24T21:07:05.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135912026-03-24T21:07:05.960ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "c9d36397-77be-4262-aca1-43852035d024", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} })
135922026-03-24T21:07:05.960ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
135932026-03-24T21:07:05.960ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
135942026-03-24T21:07:05.960ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 384.044556ms iteration = 11 status = {"errors":[],"probes_by_sled":{"b4293d4f-3308-4dfa-b261-a103215913e4":0}}
135952026-03-24T21:07:05.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135962026-03-24T21:07:05.961ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
135972026-03-24T21:07:05.961ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
135982026-03-24T21:07:05.961ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1f3978f-b7b1-4822-b4aa-85436c83e705", "content-length": "372", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} })
135992026-03-24T21:07:05.961ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
136002026-03-24T21:07:05.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
136012026-03-24T21:07:05.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136022026-03-24T21:07:05.962ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
136032026-03-24T21:07:05.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136042026-03-24T21:07:05.963ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f811f4b-cc9c-4e30-be2e-7ea1ed655806", "content-length": "691", "date": "Tue, 24 Mar 2026 21:07:05 GMT"} })
136052026-03-24T21:07:05.963ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
136062026-03-24T21:07:05.963ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
136072026-03-24T21:07:05.963ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
136082026-03-24T21:07:05.963ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 430.906469ms iteration = 21 status = {}
136092026-03-24T21:07:05.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136102026-03-24T21:07:05.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136112026-03-24T21:07:05.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136122026-03-24T21:07:05.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136132026-03-24T21:07:05.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136142026-03-24T21:07:05.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136152026-03-24T21:07:05.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136162026-03-24T21:07:05.980ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
136172026-03-24T21:07:05.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136182026-03-24T21:07:05.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136192026-03-24T21:07:05.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136202026-03-24T21:07:05.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136212026-03-24T21:07:05.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136222026-03-24T21:07:05.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136232026-03-24T21:07:05.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136242026-03-24T21:07:05.991ZDEBGnexus (BackgroundTasks): audit log timeout: no stale entries found background_task = audit_log_timeout_incomplete cutoff = 2026-03-24 17:07:05.576453078 UTC
136252026-03-24T21:07:05.991ZDEBGnexus (BackgroundTasks): activation complete background_task = audit_log_timeout_incomplete elapsed = 414.762304ms iteration = 2 status = {"cutoff":"2026-03-24T17:07:05.576453078Z","error":null,"max_timed_out_per_activation":1000,"timed_out":0}
136262026-03-24T21:07:05.994ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
136272026-03-24T21:07:05.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136282026-03-24T21:07:05.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136292026-03-24T21:07:05.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136302026-03-24T21:07:05.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136312026-03-24T21:07:06.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136322026-03-24T21:07:06.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136332026-03-24T21:07:06.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136342026-03-24T21:07:06.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136352026-03-24T21:07:06.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136362026-03-24T21:07:06.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136372026-03-24T21:07:06.011ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136382026-03-24T21:07:06.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136392026-03-24T21:07:06.013ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136402026-03-24T21:07:06.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136412026-03-24T21:07:06.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136422026-03-24T21:07:06.019ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136432026-03-24T21:07:06.020ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136442026-03-24T21:07:06.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136452026-03-24T21:07:06.021ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136462026-03-24T21:07:06.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136472026-03-24T21:07:06.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136482026-03-24T21:07:06.023ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
136492026-03-24T21:07:06.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136502026-03-24T21:07:06.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136512026-03-24T21:07:06.025ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136522026-03-24T21:07:06.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136532026-03-24T21:07:06.026ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
136542026-03-24T21:07:06.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136552026-03-24T21:07:06.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136562026-03-24T21:07:06.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136572026-03-24T21:07:06.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136582026-03-24T21:07:06.028ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
136592026-03-24T21:07:06.028ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
136602026-03-24T21:07:06.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
136612026-03-24T21:07:06.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136622026-03-24T21:07:06.029ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
136632026-03-24T21:07:06.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136642026-03-24T21:07:06.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136652026-03-24T21:07:06.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
136662026-03-24T21:07:06.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136672026-03-24T21:07:06.031ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
136682026-03-24T21:07:06.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
136692026-03-24T21:07:06.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136702026-03-24T21:07:06.035ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136712026-03-24T21:07:06.037ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
136722026-03-24T21:07:06.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136732026-03-24T21:07:06.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136742026-03-24T21:07:06.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136752026-03-24T21:07:06.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136762026-03-24T21:07:06.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136772026-03-24T21:07:06.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136782026-03-24T21:07:06.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136792026-03-24T21:07:06.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136802026-03-24T21:07:06.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136812026-03-24T21:07:06.042ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
136822026-03-24T21:07:06.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136832026-03-24T21:07:06.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136842026-03-24T21:07:06.043ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
136852026-03-24T21:07:06.043ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
136862026-03-24T21:07:06.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
136872026-03-24T21:07:06.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136882026-03-24T21:07:06.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136892026-03-24T21:07:06.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136902026-03-24T21:07:06.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136912026-03-24T21:07:06.050ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
136922026-03-24T21:07:06.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
136932026-03-24T21:07:06.053ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
136942026-03-24T21:07:06.053ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136952026-03-24T21:07:06.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136962026-03-24T21:07:06.053ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
136972026-03-24T21:07:06.053ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136982026-03-24T21:07:06.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136992026-03-24T21:07:06.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137002026-03-24T21:07:06.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137012026-03-24T21:07:06.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137022026-03-24T21:07:06.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137032026-03-24T21:07:06.058ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
137042026-03-24T21:07:06.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
137052026-03-24T21:07:06.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137062026-03-24T21:07:06.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137072026-03-24T21:07:06.061ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
137082026-03-24T21:07:06.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
137092026-03-24T21:07:06.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137102026-03-24T21:07:06.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137112026-03-24T21:07:06.063ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
137122026-03-24T21:07:06.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
137132026-03-24T21:07:06.065ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
137142026-03-24T21:07:06.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
137152026-03-24T21:07:06.066ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
137162026-03-24T21:07:06.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137172026-03-24T21:07:06.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137182026-03-24T21:07:06.067ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
137192026-03-24T21:07:06.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
137202026-03-24T21:07:06.069ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
137212026-03-24T21:07:06.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137222026-03-24T21:07:06.069ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
137232026-03-24T21:07:06.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137242026-03-24T21:07:06.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
137252026-03-24T21:07:06.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137262026-03-24T21:07:06.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137272026-03-24T21:07:06.071ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137282026-03-24T21:07:06.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137292026-03-24T21:07:06.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137302026-03-24T21:07:06.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137312026-03-24T21:07:06.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
137322026-03-24T21:07:06.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137332026-03-24T21:07:06.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137342026-03-24T21:07:06.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137352026-03-24T21:07:06.078ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
137362026-03-24T21:07:06.078ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
137372026-03-24T21:07:06.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137382026-03-24T21:07:06.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
137392026-03-24T21:07:06.078ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
137402026-03-24T21:07:06.078ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
137412026-03-24T21:07:06.079ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "bccb905a-4049-4f0f-a881-05c66d13fbf4", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
137422026-03-24T21:07:06.079ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137432026-03-24T21:07:06.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137442026-03-24T21:07:06.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137452026-03-24T21:07:06.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137462026-03-24T21:07:06.082ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137472026-03-24T21:07:06.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137482026-03-24T21:07:06.084ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
137492026-03-24T21:07:06.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
137502026-03-24T21:07:06.087ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
137512026-03-24T21:07:06.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
137522026-03-24T21:07:06.088ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137532026-03-24T21:07:06.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137542026-03-24T21:07:06.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137552026-03-24T21:07:06.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137562026-03-24T21:07:06.091ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137572026-03-24T21:07:06.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137582026-03-24T21:07:06.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137592026-03-24T21:07:06.096ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
137602026-03-24T21:07:06.096ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
137612026-03-24T21:07:06.097ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d8c7e444-1bd3-4bca-af1d-2ca4d2692b6f", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
137622026-03-24T21:07:06.097ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 536.776844ms iteration = 31 status = {}
137632026-03-24T21:07:06.097ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Dependency
137642026-03-24T21:07:06.097ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
137652026-03-24T21:07:06.097ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137662026-03-24T21:07:06.097ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
137672026-03-24T21:07:06.097ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
137682026-03-24T21:07:06.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137692026-03-24T21:07:06.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137702026-03-24T21:07:06.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
137712026-03-24T21:07:06.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137722026-03-24T21:07:06.099ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137732026-03-24T21:07:06.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137742026-03-24T21:07:06.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137752026-03-24T21:07:06.102ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
137762026-03-24T21:07:06.102ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137772026-03-24T21:07:06.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137782026-03-24T21:07:06.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137792026-03-24T21:07:06.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137802026-03-24T21:07:06.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137812026-03-24T21:07:06.104ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137822026-03-24T21:07:06.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137832026-03-24T21:07:06.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137842026-03-24T21:07:06.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137852026-03-24T21:07:06.111ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
137862026-03-24T21:07:06.111ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137872026-03-24T21:07:06.111ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
137882026-03-24T21:07:06.111ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
137892026-03-24T21:07:06.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137902026-03-24T21:07:06.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
137912026-03-24T21:07:06.113ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137922026-03-24T21:07:06.115ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
137932026-03-24T21:07:06.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137942026-03-24T21:07:06.115ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
137952026-03-24T21:07:06.115ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
137962026-03-24T21:07:06.115ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
137972026-03-24T21:07:06.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137982026-03-24T21:07:06.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137992026-03-24T21:07:06.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
138002026-03-24T21:07:06.117ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
138012026-03-24T21:07:06.117ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
138022026-03-24T21:07:06.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
138032026-03-24T21:07:06.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138042026-03-24T21:07:06.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
138052026-03-24T21:07:06.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
138062026-03-24T21:07:06.119ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b176f0dd-c2a3-4a11-9569-f0dd6c228dae", "content-length": "372", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
138072026-03-24T21:07:06.119ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
138082026-03-24T21:07:06.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
138092026-03-24T21:07:06.119ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
138102026-03-24T21:07:06.119ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
138112026-03-24T21:07:06.119ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
138122026-03-24T21:07:06.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
138132026-03-24T21:07:06.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
138142026-03-24T21:07:06.121ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
138152026-03-24T21:07:06.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138162026-03-24T21:07:06.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138172026-03-24T21:07:06.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138182026-03-24T21:07:06.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138192026-03-24T21:07:06.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138202026-03-24T21:07:06.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138212026-03-24T21:07:06.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138222026-03-24T21:07:06.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138232026-03-24T21:07:06.129ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
138242026-03-24T21:07:06.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
138252026-03-24T21:07:06.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
138262026-03-24T21:07:06.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
138272026-03-24T21:07:06.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138282026-03-24T21:07:06.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138292026-03-24T21:07:06.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138302026-03-24T21:07:06.136ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
138312026-03-24T21:07:06.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138322026-03-24T21:07:06.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138332026-03-24T21:07:06.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138342026-03-24T21:07:06.143ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
138352026-03-24T21:07:06.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
138362026-03-24T21:07:06.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138372026-03-24T21:07:06.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
138382026-03-24T21:07:06.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138392026-03-24T21:07:06.144ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
138402026-03-24T21:07:06.144ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
138412026-03-24T21:07:06.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138422026-03-24T21:07:06.145ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4264a589-c373-4d72-92b2-066a195f1f26", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
138432026-03-24T21:07:06.145ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
138442026-03-24T21:07:06.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138452026-03-24T21:07:06.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
138462026-03-24T21:07:06.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138472026-03-24T21:07:06.148ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
138482026-03-24T21:07:06.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
138492026-03-24T21:07:06.148ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
138502026-03-24T21:07:06.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
138512026-03-24T21:07:06.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
138522026-03-24T21:07:06.150ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
138532026-03-24T21:07:06.150ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
138542026-03-24T21:07:06.150ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
138552026-03-24T21:07:06.151ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
138562026-03-24T21:07:06.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
138572026-03-24T21:07:06.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
138582026-03-24T21:07:06.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138592026-03-24T21:07:06.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138602026-03-24T21:07:06.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138612026-03-24T21:07:06.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138622026-03-24T21:07:06.155ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
138632026-03-24T21:07:06.155ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
138642026-03-24T21:07:06.156ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b2f987b9-f06f-4a18-85e1-2bb2f3c3b556", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
138652026-03-24T21:07:06.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138662026-03-24T21:07:06.156ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 59.075593ms iteration = 32 status = {}
138672026-03-24T21:07:06.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
138682026-03-24T21:07:06.164ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:118 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
138692026-03-24T21:07:06.164ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 604.345607ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
138702026-03-24T21:07:06.201ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d7b0c75c-9e5f-4030-b3c2-fe2659e0344f", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
138712026-03-24T21:07:06.202ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
138722026-03-24T21:07:06.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138732026-03-24T21:07:06.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138742026-03-24T21:07:06.219ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
138752026-03-24T21:07:06.219ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
138762026-03-24T21:07:06.220ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
138772026-03-24T21:07:06.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
138782026-03-24T21:07:06.225ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
138792026-03-24T21:07:06.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
138802026-03-24T21:07:06.228ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138812026-03-24T21:07:06.233ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
138822026-03-24T21:07:06.233ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
138832026-03-24T21:07:06.234ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "ee48038d-44d7-4684-9a0a-0a94bf31e45c", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
138842026-03-24T21:07:06.234ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:506
138852026-03-24T21:07:06.234ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 678.800854ms
138862026-03-24T21:07:06.234ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 678.816304ms iteration = 3 status = {}
138872026-03-24T21:07:06.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138882026-03-24T21:07:06.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138892026-03-24T21:07:06.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138902026-03-24T21:07:06.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138912026-03-24T21:07:06.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138922026-03-24T21:07:06.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
138932026-03-24T21:07:06.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138942026-03-24T21:07:06.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138952026-03-24T21:07:06.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138962026-03-24T21:07:06.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138972026-03-24T21:07:06.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138982026-03-24T21:07:06.253ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
138992026-03-24T21:07:06.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139002026-03-24T21:07:06.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139012026-03-24T21:07:06.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139022026-03-24T21:07:06.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139032026-03-24T21:07:06.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139042026-03-24T21:07:06.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
139052026-03-24T21:07:06.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139062026-03-24T21:07:06.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139072026-03-24T21:07:06.267ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
139082026-03-24T21:07:06.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
139092026-03-24T21:07:06.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139102026-03-24T21:07:06.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139112026-03-24T21:07:06.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139122026-03-24T21:07:06.270ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
139132026-03-24T21:07:06.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
139142026-03-24T21:07:06.271ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
139152026-03-24T21:07:06.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139162026-03-24T21:07:06.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139172026-03-24T21:07:06.272ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
139182026-03-24T21:07:06.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
139192026-03-24T21:07:06.274ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 742.415899ms iteration = 21 status = {}
139202026-03-24T21:07:06.319ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
139212026-03-24T21:07:06.319ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
139222026-03-24T21:07:06.319ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = be6e76c5-cd9d-4c51-aa6b-d6c859eb20bb parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
139232026-03-24T21:07:06.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139242026-03-24T21:07:06.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139252026-03-24T21:07:06.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139262026-03-24T21:07:06.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
139272026-03-24T21:07:06.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139282026-03-24T21:07:06.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139292026-03-24T21:07:06.327ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
139302026-03-24T21:07:06.327ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
139312026-03-24T21:07:06.327ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 795.018038ms iteration = 26 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
139322026-03-24T21:07:06.348ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
139332026-03-24T21:07:06.348ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
139342026-03-24T21:07:06.348ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139352026-03-24T21:07:06.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139362026-03-24T21:07:06.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139372026-03-24T21:07:06.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139382026-03-24T21:07:06.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139392026-03-24T21:07:06.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
139402026-03-24T21:07:06.357ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
139412026-03-24T21:07:06.358ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "44f5f8be-e3c8-4dfe-879b-3d2be2011c3e", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
139422026-03-24T21:07:06.358ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
139432026-03-24T21:07:06.358ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = Dns step id = Ensure
139442026-03-24T21:07:06.358ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
139452026-03-24T21:07:06.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139462026-03-24T21:07:06.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139472026-03-24T21:07:06.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139482026-03-24T21:07:06.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139492026-03-24T21:07:06.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139502026-03-24T21:07:06.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
139512026-03-24T21:07:06.366ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
139522026-03-24T21:07:06.366ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
139532026-03-24T21:07:06.366ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
139542026-03-24T21:07:06.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139552026-03-24T21:07:06.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139562026-03-24T21:07:06.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139572026-03-24T21:07:06.371ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
139582026-03-24T21:07:06.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139592026-03-24T21:07:06.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139602026-03-24T21:07:06.373ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
139612026-03-24T21:07:06.373ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
139622026-03-24T21:07:06.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139632026-03-24T21:07:06.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139642026-03-24T21:07:06.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139652026-03-24T21:07:06.378ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
139662026-03-24T21:07:06.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139672026-03-24T21:07:06.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139682026-03-24T21:07:06.381ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
139692026-03-24T21:07:06.381ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
139702026-03-24T21:07:06.381ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
139712026-03-24T21:07:06.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139722026-03-24T21:07:06.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139732026-03-24T21:07:06.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139742026-03-24T21:07:06.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139752026-03-24T21:07:06.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139762026-03-24T21:07:06.389ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
139772026-03-24T21:07:06.436ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
139782026-03-24T21:07:06.436ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
139792026-03-24T21:07:06.436ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
139802026-03-24T21:07:06.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139812026-03-24T21:07:06.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139822026-03-24T21:07:06.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139832026-03-24T21:07:06.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
139842026-03-24T21:07:06.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139852026-03-24T21:07:06.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139862026-03-24T21:07:06.444ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
139872026-03-24T21:07:06.444ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
139882026-03-24T21:07:06.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139892026-03-24T21:07:06.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139902026-03-24T21:07:06.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139912026-03-24T21:07:06.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
139922026-03-24T21:07:06.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139932026-03-24T21:07:06.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139942026-03-24T21:07:06.452ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
139952026-03-24T21:07:06.452ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
139962026-03-24T21:07:06.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139972026-03-24T21:07:06.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139982026-03-24T21:07:06.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139992026-03-24T21:07:06.459ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
140002026-03-24T21:07:06.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140012026-03-24T21:07:06.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140022026-03-24T21:07:06.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140032026-03-24T21:07:06.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140042026-03-24T21:07:06.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140052026-03-24T21:07:06.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140062026-03-24T21:07:06.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140072026-03-24T21:07:06.468ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
140082026-03-24T21:07:06.469ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
140092026-03-24T21:07:06.469ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
140102026-03-24T21:07:06.470ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = OmicronZones step id = Cleanup
140112026-03-24T21:07:06.470ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = OmicronZones step id = Cleanup
140122026-03-24T21:07:06.470ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = PhysicalDisks step id = Cleanup
140132026-03-24T21:07:06.470ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = Clickhouse step id = Ensure
140142026-03-24T21:07:06.470ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = Clickhouse step id = Ensure
140152026-03-24T21:07:06.470ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
140162026-03-24T21:07:06.476ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "5a95799b-c5f3-4103-8c35-d810111b67d6", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
140172026-03-24T21:07:06.476ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = SupportBundles step id = Cleanup
140182026-03-24T21:07:06.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140192026-03-24T21:07:06.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140202026-03-24T21:07:06.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140212026-03-24T21:07:06.479ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
140222026-03-24T21:07:06.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140232026-03-24T21:07:06.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140242026-03-24T21:07:06.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140252026-03-24T21:07:06.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140262026-03-24T21:07:06.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140272026-03-24T21:07:06.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
140282026-03-24T21:07:06.490ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = OmicronZones step id = Cleanup
140292026-03-24T21:07:06.490ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
140302026-03-24T21:07:06.490ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
140312026-03-24T21:07:06.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140322026-03-24T21:07:06.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140332026-03-24T21:07:06.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140342026-03-24T21:07:06.493ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
140352026-03-24T21:07:06.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140362026-03-24T21:07:06.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140372026-03-24T21:07:06.495ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
140382026-03-24T21:07:06.495ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9e1c09c1-3960-4ecf-a904-19e137699ab3 (blueprint), false)
140392026-03-24T21:07:06.495ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = Cockroach step id = Ensure
140402026-03-24T21:07:06.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140412026-03-24T21:07:06.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140422026-03-24T21:07:06.526ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
140432026-03-24T21:07:06.526ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c05a639c-a0af-40b1-9cb2-99b6da88fed6 step component = MgsUpdates step id = Ensure
140442026-03-24T21:07:06.526ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 13 reason = Dependency
140452026-03-24T21:07:06.526ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 32 reason = Dependency
140462026-03-24T21:07:06.526ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140472026-03-24T21:07:06.526ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
140482026-03-24T21:07:06.527ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.005339538s iteration = 12 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"94830173-cb89-4a61-ac89-6713132f688e","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3270,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":68844666,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":68844616,"secs":0}},"event_index":1,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":68857967,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":197046144,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":197046094,"secs":0}},"event_index":2,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":265921061,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":145628417,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":145628377,"secs":0}},"event_index":3,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":411565798,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":23826023,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":23825983,"secs":0}},"event_index":4,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":435402291,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":401487186,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":401487156,"secs":0}},"event_index":5,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":836933058,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":111341002,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":111340962,"secs":0}},"event_index":6,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":948281160,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5810,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5770,"secs":0}},"event_index":7,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":948300970,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2330,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2290,"secs":0}},"event_index":8,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":948305610,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3920,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3880,"secs":0}},"event_index":9,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":948311790,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1800,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1770,"secs":0}},"event_index":10,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":948315011,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6348991,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":6348961,"secs":0}},"event_index":11,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":954665162,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":13990097,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":13990057,"secs":0}},"event_index":12,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":968661479,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5000805,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5000755,"secs":0}},"event_index":13,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":973679165,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":31331005,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":31330965,"secs":0}},"event_index":14,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":5019920,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5540,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5500,"secs":0}},"event_index":15,"execution_id":"94830173-cb89-4a61-ac89-6713132f688e","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":5034280,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3"}
140492026-03-24T21:07:06.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
140502026-03-24T21:07:06.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
140512026-03-24T21:07:06.528ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140522026-03-24T21:07:06.529ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140532026-03-24T21:07:06.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
140542026-03-24T21:07:06.531ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
140552026-03-24T21:07:06.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
140562026-03-24T21:07:06.532ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
140572026-03-24T21:07:06.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140582026-03-24T21:07:06.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
140592026-03-24T21:07:06.533ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
140602026-03-24T21:07:06.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
140612026-03-24T21:07:06.535ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140622026-03-24T21:07:06.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
140632026-03-24T21:07:06.537ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
140642026-03-24T21:07:06.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
140652026-03-24T21:07:06.539ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
140662026-03-24T21:07:06.539ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.576129ms iteration = 32 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
140672026-03-24T21:07:06.608ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(168e3635-4476-461b-bf0f-55f7661feb42, 0), (d8778b8e-61f7-48de-bcc7-ba6d2534210e, 0), (df496995-f9d8-4235-b1db-c1cf67df3a52, 0), (65047d2b-6548-41c3-a0e1-ec24af6ef052, 0)] collection_id = 168e3635-4476-461b-bf0f-55f7661feb42
140682026-03-24T21:07:06.608ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140692026-03-24T21:07:06.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
140702026-03-24T21:07:06.797ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 168e3635-4476-461b-bf0f-55f7661feb42 file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
140712026-03-24T21:07:06.797ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140722026-03-24T21:07:06.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
140732026-03-24T21:07:06.903ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(df496995-f9d8-4235-b1db-c1cf67df3a52, 0), (65047d2b-6548-41c3-a0e1-ec24af6ef052, 0), (77fe5a15-f20f-4e49-88d4-56187bc521a7, 0)]
140742026-03-24T21:07:06.903ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1413, tv_nsec: 117743544 } })
140752026-03-24T21:07:06.906ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1345db7a-abc1-4a4c-a49c-e6c56102e521.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1920db37-87f5-4a96-b3ef-1db2eb59e9a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6b02d2eb-4373-4f12-8efc-4c0e3f4fa77b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("709c4704-f768-4640-a198-a23621ed6e11.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1413, tv_nsec: 119851941 } })
140762026-03-24T21:07:06.906ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
140772026-03-24T21:07:06.906ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
140782026-03-24T21:07:06.906ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
140792026-03-24T21:07:06.907ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "629e6881-fa0c-4374-8618-8b7611e04cb1", "content-length": "476", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
140802026-03-24T21:07:06.907ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
140812026-03-24T21:07:06.907ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fa5230c-2d03-44fc-90a8-08ca70c83c66", "content-length": "734", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
140822026-03-24T21:07:06.907ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
140832026-03-24T21:07:06.908ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef32537b-2e4c-49d7-9ee7-378bb2ed9883", "content-length": "111", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
140842026-03-24T21:07:06.908ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
140852026-03-24T21:07:06.908ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "53296ce9-eb87-4d14-9bb8-69890717a906", "content-length": "111", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
140862026-03-24T21:07:06.908ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
140872026-03-24T21:07:06.908ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "598d4f93-0b25-49a4-ba61-1e834abbc8d4", "content-length": "163", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
140882026-03-24T21:07:06.909ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
140892026-03-24T21:07:06.909ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b867424-4b3d-4586-b22d-0d38a30277e9", "content-length": "163", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
140902026-03-24T21:07:06.909ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
140912026-03-24T21:07:06.909ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f11db1cc-7fd1-4359-9838-f0b25cdcefd5", "content-length": "166", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
140922026-03-24T21:07:06.909ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
140932026-03-24T21:07:06.910ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "428210d8-2cb4-4b94-9fd6-9e5c939240dc", "content-length": "166", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
140942026-03-24T21:07:06.910ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
140952026-03-24T21:07:06.910ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ec59d42-c20d-4c44-96ef-c165a23b63e7", "content-length": "702", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
140962026-03-24T21:07:06.910ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
140972026-03-24T21:07:06.910ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d1e4896-f2cd-4979-848d-dbb7db2b758a", "content-length": "727", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
140982026-03-24T21:07:06.910ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
140992026-03-24T21:07:06.911ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e6b4263-24f9-4bf9-83be-f1793fb7e1e4", "content-length": "729", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141002026-03-24T21:07:06.911ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
141012026-03-24T21:07:06.911ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "16dbf02e-d752-4c15-b76e-eeec64272628", "content-length": "728", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141022026-03-24T21:07:06.911ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
141032026-03-24T21:07:06.911ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "728317d7-9b0f-4115-aa86-0e480dcf55c5", "content-length": "722", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141042026-03-24T21:07:06.911ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
141052026-03-24T21:07:06.912ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc5c1417-c37d-43ad-9d01-1dcdc833ed5f", "content-length": "10", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141062026-03-24T21:07:06.912ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
141072026-03-24T21:07:06.912ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6423cb4e-f8df-420e-bc45-ebf62c4a2d04", "content-length": "144", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141082026-03-24T21:07:06.912ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
141092026-03-24T21:07:06.912ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e895ddc-6910-4c87-bd71-9c4dc614912a", "content-length": "147", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141102026-03-24T21:07:06.912ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
141112026-03-24T21:07:06.913ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d6c8d11-943b-4506-9f88-973c9531f6bb", "content-length": "109", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141122026-03-24T21:07:06.913ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
141132026-03-24T21:07:06.913ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1543fc85-f3ee-453a-9b15-ece7d4b746ad", "content-length": "109", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141142026-03-24T21:07:06.913ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
141152026-03-24T21:07:06.914ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4816205e-0732-4821-99b5-2ca4cd0c5307", "content-length": "163", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141162026-03-24T21:07:06.914ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
141172026-03-24T21:07:06.914ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed25015d-eb3d-40f0-8389-973dd2c04484", "content-length": "163", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141182026-03-24T21:07:06.914ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
141192026-03-24T21:07:06.914ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8f524b1-00e2-460f-b85a-de89671d8740", "content-length": "166", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141202026-03-24T21:07:06.914ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
141212026-03-24T21:07:06.915ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c1f18b7-6003-41f1-ba75-46b609d65754", "content-length": "166", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141222026-03-24T21:07:06.915ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
141232026-03-24T21:07:06.915ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "634c3a25-487c-457f-a8bc-8090f7983f93", "content-length": "702", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141242026-03-24T21:07:06.915ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
141252026-03-24T21:07:06.915ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ac7a4a5-0b04-4967-bd0f-c65b680a30aa", "content-length": "727", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141262026-03-24T21:07:06.915ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
141272026-03-24T21:07:06.916ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "96d17dc2-ae6d-45fd-a5c2-1db1a3ccff25", "content-length": "729", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141282026-03-24T21:07:06.916ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
141292026-03-24T21:07:06.916ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b6a20e4-e5a8-4f7f-b5a7-fa6e8cbd452c", "content-length": "728", "date": "Tue, 24 Mar 2026 21:07:06 GMT"} })
141302026-03-24T21:07:06.916ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141312026-03-24T21:07:06.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
141322026-03-24T21:07:06.919ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141332026-03-24T21:07:06.921ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
141342026-03-24T21:07:06.921ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141352026-03-24T21:07:06.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
141362026-03-24T21:07:06.922ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141372026-03-24T21:07:06.924ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
141382026-03-24T21:07:06.924ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141392026-03-24T21:07:06.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
141402026-03-24T21:07:06.930ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
141412026-03-24T21:07:06.930ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
141422026-03-24T21:07:07.021ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "977aa558-a0ba-4ece-8e1c-6a507832e991", "content-length": "110672", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141432026-03-24T21:07:07.022ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
141442026-03-24T21:07:07.022ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
141452026-03-24T21:07:07.022ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
141462026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
141472026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
141482026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
141492026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
141502026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
141512026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "87e12e95-6894-4b34-a32d-46d34771326f", "content-length": "64", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141522026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "00490b97-9c6f-4895-a679-67fe1b3c7b42", "content-length": "64", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141532026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
141542026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "40d93839-0204-4a98-ad7a-42d039ebb474", "content-length": "64", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141552026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f06a89a-268e-46fa-94ae-c63368acee38", "content-length": "64", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141562026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
141572026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
141582026-03-24T21:07:07.022ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
141592026-03-24T21:07:07.023ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "616aba14-0d39-4897-b252-605d7de021df", "content-length": "64", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141602026-03-24T21:07:07.023ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
141612026-03-24T21:07:07.030ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "26e55fb6-3fe2-4078-95db-6ab5ad419b9c", "content-length": "402986", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141622026-03-24T21:07:07.030ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "4be5ad2d-d885-4b90-819a-a41ae7c9b8c9", "content-length": "406289", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141632026-03-24T21:07:07.031ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "76cae56e-7cca-4d5d-9059-c074d540866f", "content-length": "406735", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141642026-03-24T21:07:07.032ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a5477e0-b2b3-4441-977c-4c8eba9f8d52", "content-length": "434915", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141652026-03-24T21:07:07.033ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb75670d-17dc-45d0-a00e-3e435736fcaa", "content-length": "415530", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141662026-03-24T21:07:07.037ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
141672026-03-24T21:07:07.037ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
141682026-03-24T21:07:07.037ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
141692026-03-24T21:07:07.037ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
141702026-03-24T21:07:07.037ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
141712026-03-24T21:07:07.039ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 (omicron_zone)
141722026-03-24T21:07:07.039ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
141732026-03-24T21:07:07.042ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fa8ca36-cec6-468b-a435-127b6e3cb664", "content-length": "128", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141742026-03-24T21:07:07.042ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
141752026-03-24T21:07:07.045ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
141762026-03-24T21:07:07.045ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
141772026-03-24T21:07:07.045ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c1a10b4-2863-4611-95f1-f234f05c2461", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141782026-03-24T21:07:07.046ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
141792026-03-24T21:07:07.046ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
141802026-03-24T21:07:07.046ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
141812026-03-24T21:07:07.046ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad8792d5-00b4-493a-8d61-0abd19b038ee", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141822026-03-24T21:07:07.046ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
141832026-03-24T21:07:07.046ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
141842026-03-24T21:07:07.046ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
141852026-03-24T21:07:07.047ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b6fd993-1a15-4bda-90eb-0828877f7b3f", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:07:07 GMT"} })
141862026-03-24T21:07:07.047ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
141872026-03-24T21:07:07.047ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
141882026-03-24T21:07:07.047ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
141892026-03-24T21:07:07.047ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141902026-03-24T21:07:07.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
141912026-03-24T21:07:07.049ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141922026-03-24T21:07:07.052ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
141932026-03-24T21:07:07.052ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141942026-03-24T21:07:07.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
141952026-03-24T21:07:07.277ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = da877b83-376f-4ce2-a60a-42f1dd57b87c file = nexus/db-queries/src/db/datastore/inventory.rs:1757
141962026-03-24T21:07:07.277ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = da877b83-376f-4ce2-a60a-42f1dd57b87c time_started = 2026-03-24 21:07:06.906741 UTC
141972026-03-24T21:07:07.277ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 54 reason = Dependency
141982026-03-24T21:07:07.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
141992026-03-24T21:07:07.277ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 750.535478ms iteration = 13 status = {"collection_id":"da877b83-376f-4ce2-a60a-42f1dd57b87c","time_done":"2026-03-24 21:07:07.047363 UTC","time_started":"2026-03-24 21:07:06.906741 UTC"}
142002026-03-24T21:07:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
142012026-03-24T21:07:07.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142022026-03-24T21:07:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
142032026-03-24T21:07:07.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
142042026-03-24T21:07:07.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
142052026-03-24T21:07:07.315ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
142062026-03-24T21:07:07.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
142072026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 25 reason = Dependency
142082026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 97.945481ms iteration = 54 status = {"Loaded":{"collection_id":"da877b83-376f-4ce2-a60a-42f1dd57b87c","time_loaded":"2026-03-24T21:07:07.277337672Z","time_started":"2026-03-24T21:07:06.906741Z"}}
142092026-03-24T21:07:07.375ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
142102026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 27 reason = Dependency
142112026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 35 reason = Dependency
142122026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 18.88\u{b5}s iteration = 25 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
142132026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142142026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 17 reason = Dependency
142152026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
142162026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142172026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
142182026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142192026-03-24T21:07:07.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142202026-03-24T21:07:07.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142212026-03-24T21:07:07.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142222026-03-24T21:07:07.377ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142232026-03-24T21:07:07.379ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
142242026-03-24T21:07:07.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142252026-03-24T21:07:07.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142262026-03-24T21:07:07.380ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
142272026-03-24T21:07:07.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142282026-03-24T21:07:07.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142292026-03-24T21:07:07.380ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
142302026-03-24T21:07:07.380ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
142312026-03-24T21:07:07.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
142322026-03-24T21:07:07.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142332026-03-24T21:07:07.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142342026-03-24T21:07:07.382ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
142352026-03-24T21:07:07.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
142362026-03-24T21:07:07.383ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
142372026-03-24T21:07:07.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142382026-03-24T21:07:07.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142392026-03-24T21:07:07.384ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
142402026-03-24T21:07:07.384ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142412026-03-24T21:07:07.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142422026-03-24T21:07:07.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142432026-03-24T21:07:07.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142442026-03-24T21:07:07.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142452026-03-24T21:07:07.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142462026-03-24T21:07:07.386ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
142472026-03-24T21:07:07.386ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
142482026-03-24T21:07:07.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142492026-03-24T21:07:07.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
142502026-03-24T21:07:07.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142512026-03-24T21:07:07.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142522026-03-24T21:07:07.390ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
142532026-03-24T21:07:07.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142542026-03-24T21:07:07.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142552026-03-24T21:07:07.391ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
142562026-03-24T21:07:07.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142572026-03-24T21:07:07.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142582026-03-24T21:07:07.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142592026-03-24T21:07:07.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142602026-03-24T21:07:07.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
142612026-03-24T21:07:07.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142622026-03-24T21:07:07.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142632026-03-24T21:07:07.395ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
142642026-03-24T21:07:07.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142652026-03-24T21:07:07.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142662026-03-24T21:07:07.396ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.619972ms iteration = 35 status = {"physical_disks_added":0}
142672026-03-24T21:07:07.399ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
142682026-03-24T21:07:07.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142692026-03-24T21:07:07.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142702026-03-24T21:07:07.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142712026-03-24T21:07:07.403ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
142722026-03-24T21:07:07.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142732026-03-24T21:07:07.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142742026-03-24T21:07:07.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142752026-03-24T21:07:07.407ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
142762026-03-24T21:07:07.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142772026-03-24T21:07:07.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142782026-03-24T21:07:07.409ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
142792026-03-24T21:07:07.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142802026-03-24T21:07:07.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142812026-03-24T21:07:07.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142822026-03-24T21:07:07.414ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
142832026-03-24T21:07:07.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142842026-03-24T21:07:07.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142852026-03-24T21:07:07.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142862026-03-24T21:07:07.419ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
142872026-03-24T21:07:07.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
142882026-03-24T21:07:07.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
142892026-03-24T21:07:07.420ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
142902026-03-24T21:07:07.420ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 45.2364ms iteration = 17 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"da877b83-376f-4ce2-a60a-42f1dd57b87c","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
142912026-03-24T21:07:07.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142922026-03-24T21:07:07.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142932026-03-24T21:07:07.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142942026-03-24T21:07:07.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142952026-03-24T21:07:07.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142962026-03-24T21:07:07.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
142972026-03-24T21:07:07.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142982026-03-24T21:07:07.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142992026-03-24T21:07:07.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143002026-03-24T21:07:07.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143012026-03-24T21:07:07.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143022026-03-24T21:07:07.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
143032026-03-24T21:07:07.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143042026-03-24T21:07:07.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143052026-03-24T21:07:07.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143062026-03-24T21:07:07.464ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
143072026-03-24T21:07:07.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143082026-03-24T21:07:07.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143092026-03-24T21:07:07.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143102026-03-24T21:07:07.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143112026-03-24T21:07:07.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143122026-03-24T21:07:07.470ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
143132026-03-24T21:07:07.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143142026-03-24T21:07:07.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143152026-03-24T21:07:07.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143162026-03-24T21:07:07.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143172026-03-24T21:07:07.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143182026-03-24T21:07:07.476ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
143192026-03-24T21:07:07.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143202026-03-24T21:07:07.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143212026-03-24T21:07:07.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143222026-03-24T21:07:07.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143232026-03-24T21:07:07.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143242026-03-24T21:07:07.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
143252026-03-24T21:07:07.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143262026-03-24T21:07:07.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143272026-03-24T21:07:07.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143282026-03-24T21:07:07.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143292026-03-24T21:07:07.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143302026-03-24T21:07:07.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143312026-03-24T21:07:07.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143322026-03-24T21:07:07.506ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
143332026-03-24T21:07:07.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143342026-03-24T21:07:07.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143352026-03-24T21:07:07.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143362026-03-24T21:07:07.510ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
143372026-03-24T21:07:07.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143382026-03-24T21:07:07.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143392026-03-24T21:07:07.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143402026-03-24T21:07:07.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143412026-03-24T21:07:07.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143422026-03-24T21:07:07.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143432026-03-24T21:07:07.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143442026-03-24T21:07:07.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
143452026-03-24T21:07:07.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143462026-03-24T21:07:07.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143472026-03-24T21:07:07.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143482026-03-24T21:07:07.523ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
143492026-03-24T21:07:07.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143502026-03-24T21:07:07.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143512026-03-24T21:07:07.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143522026-03-24T21:07:07.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143532026-03-24T21:07:07.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143542026-03-24T21:07:07.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143552026-03-24T21:07:07.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143562026-03-24T21:07:07.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
143572026-03-24T21:07:07.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143582026-03-24T21:07:07.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143592026-03-24T21:07:07.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143602026-03-24T21:07:07.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143612026-03-24T21:07:07.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143622026-03-24T21:07:07.536ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
143632026-03-24T21:07:07.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143642026-03-24T21:07:07.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143652026-03-24T21:07:07.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143662026-03-24T21:07:07.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143672026-03-24T21:07:07.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143682026-03-24T21:07:07.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
143692026-03-24T21:07:07.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143702026-03-24T21:07:07.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143712026-03-24T21:07:07.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143722026-03-24T21:07:07.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143732026-03-24T21:07:07.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143742026-03-24T21:07:07.589ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
143752026-03-24T21:07:07.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143762026-03-24T21:07:07.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143772026-03-24T21:07:07.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143782026-03-24T21:07:07.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143792026-03-24T21:07:07.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143802026-03-24T21:07:07.596ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
143812026-03-24T21:07:07.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143822026-03-24T21:07:07.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143832026-03-24T21:07:07.598ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143842026-03-24T21:07:07.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143852026-03-24T21:07:07.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143862026-03-24T21:07:07.601ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
143872026-03-24T21:07:07.601ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143882026-03-24T21:07:07.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143892026-03-24T21:07:07.649ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
143902026-03-24T21:07:07.649ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
143912026-03-24T21:07:07.650ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 88f02502-fd31-41c6-9737-e5b09bb0821f parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
143922026-03-24T21:07:07.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143932026-03-24T21:07:07.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143942026-03-24T21:07:07.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143952026-03-24T21:07:07.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
143962026-03-24T21:07:07.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
143972026-03-24T21:07:07.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
143982026-03-24T21:07:07.657ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
143992026-03-24T21:07:07.657ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
144002026-03-24T21:07:07.657ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 282.25402ms iteration = 27 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
144012026-03-24T21:07:08.551ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = ec6335f8-9823-4c05-9fcb-6b837da9dd9b uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
144022026-03-24T21:07:08.551ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = ec6335f8-9823-4c05-9fcb-6b837da9dd9b roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
144032026-03-24T21:07:08.552ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = ec6335f8-9823-4c05-9fcb-6b837da9dd9b resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
144042026-03-24T21:07:08.553ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 1986 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = ec6335f8-9823-4c05-9fcb-6b837da9dd9b response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
144052026-03-24T21:07:08.554ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 8d0dc5a4-6330-4831-ae79-a66e2a870dba uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
144062026-03-24T21:07:08.554ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 8d0dc5a4-6330-4831-ae79-a66e2a870dba roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
144072026-03-24T21:07:08.554ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 8d0dc5a4-6330-4831-ae79-a66e2a870dba resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
144082026-03-24T21:07:08.556ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2145 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 8d0dc5a4-6330-4831-ae79-a66e2a870dba response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
144092026-03-24T21:07:08.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 335 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = d53dccd1-7d6f-436a-894f-77c4d514f421 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
144102026-03-24T21:07:09.564ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 14 reason = Timeout
144112026-03-24T21:07:09.564ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144122026-03-24T21:07:09.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144132026-03-24T21:07:09.565ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144142026-03-24T21:07:09.568ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
144152026-03-24T21:07:09.568ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144162026-03-24T21:07:09.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144172026-03-24T21:07:09.572ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(df496995-f9d8-4235-b1db-c1cf67df3a52, 0), (65047d2b-6548-41c3-a0e1-ec24af6ef052, 0), (77fe5a15-f20f-4e49-88d4-56187bc521a7, 0), (ca5beb68-7cb6-439a-97b4-01be938ee454, 0)] collection_id = df496995-f9d8-4235-b1db-c1cf67df3a52
144182026-03-24T21:07:09.572ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144192026-03-24T21:07:09.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144202026-03-24T21:07:09.719ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = df496995-f9d8-4235-b1db-c1cf67df3a52 file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
144212026-03-24T21:07:09.719ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144222026-03-24T21:07:09.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144232026-03-24T21:07:09.724ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(65047d2b-6548-41c3-a0e1-ec24af6ef052, 0), (77fe5a15-f20f-4e49-88d4-56187bc521a7, 0), (ca5beb68-7cb6-439a-97b4-01be938ee454, 0), (790f3085-8870-4f41-8538-c1fd8911c833, 0)] collection_id = 65047d2b-6548-41c3-a0e1-ec24af6ef052
144242026-03-24T21:07:09.724ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144252026-03-24T21:07:09.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144262026-03-24T21:07:09.893ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 65047d2b-6548-41c3-a0e1-ec24af6ef052 file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
144272026-03-24T21:07:09.893ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144282026-03-24T21:07:09.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144292026-03-24T21:07:09.902ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(77fe5a15-f20f-4e49-88d4-56187bc521a7, 0), (ca5beb68-7cb6-439a-97b4-01be938ee454, 0), (790f3085-8870-4f41-8538-c1fd8911c833, 0), (da877b83-376f-4ce2-a60a-42f1dd57b87c, 0)] collection_id = 77fe5a15-f20f-4e49-88d4-56187bc521a7
144302026-03-24T21:07:09.902ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144312026-03-24T21:07:09.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144322026-03-24T21:07:10.065ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 77fe5a15-f20f-4e49-88d4-56187bc521a7 file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
144332026-03-24T21:07:10.065ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144342026-03-24T21:07:10.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144352026-03-24T21:07:10.071ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(ca5beb68-7cb6-439a-97b4-01be938ee454, 0), (790f3085-8870-4f41-8538-c1fd8911c833, 0), (da877b83-376f-4ce2-a60a-42f1dd57b87c, 0)]
144362026-03-24T21:07:10.072ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1416, tv_nsec: 286369513 } })
144372026-03-24T21:07:10.074ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1345db7a-abc1-4a4c-a49c-e6c56102e521.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1920db37-87f5-4a96-b3ef-1db2eb59e9a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6b02d2eb-4373-4f12-8efc-4c0e3f4fa77b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("709c4704-f768-4640-a198-a23621ed6e11.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1416, tv_nsec: 288486980 } })
144382026-03-24T21:07:10.075ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
144392026-03-24T21:07:10.075ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
144402026-03-24T21:07:10.075ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
144412026-03-24T21:07:10.076ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "1438907f-e5a9-4a70-878b-e5c952dbb063", "content-length": "476", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144422026-03-24T21:07:10.076ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
144432026-03-24T21:07:10.076ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd4ef514-e33d-4e56-a9b8-9a9f44abaa33", "content-length": "734", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144442026-03-24T21:07:10.076ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
144452026-03-24T21:07:10.077ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4097995-bfec-46aa-914e-30fc099a493c", "content-length": "111", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144462026-03-24T21:07:10.077ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
144472026-03-24T21:07:10.078ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e6f3578-8cc4-4215-b484-c03e7ef71d2d", "content-length": "111", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144482026-03-24T21:07:10.078ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
144492026-03-24T21:07:10.078ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7127758d-b930-493b-877b-449294ec56ce", "content-length": "163", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144502026-03-24T21:07:10.078ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
144512026-03-24T21:07:10.079ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e284952f-4ad3-429f-8160-57956cc11a3b", "content-length": "163", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144522026-03-24T21:07:10.079ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
144532026-03-24T21:07:10.079ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "69e37f54-c19e-46d4-817f-29928e1f99f6", "content-length": "166", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144542026-03-24T21:07:10.079ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
144552026-03-24T21:07:10.080ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "022f85ef-dd60-4f39-8234-cc2b350a2334", "content-length": "166", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144562026-03-24T21:07:10.080ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
144572026-03-24T21:07:10.080ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b74f96a7-871b-47da-aaaf-0fffe77c681f", "content-length": "702", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144582026-03-24T21:07:10.080ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
144592026-03-24T21:07:10.080ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d5c66bc-9732-4943-86d0-fe835ee7aed4", "content-length": "727", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144602026-03-24T21:07:10.080ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
144612026-03-24T21:07:10.081ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "47bea617-d634-411e-bb7c-cf8f8c992008", "content-length": "729", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144622026-03-24T21:07:10.081ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
144632026-03-24T21:07:10.081ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2d40179-7e07-416d-8423-1be8f53ef6b6", "content-length": "728", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144642026-03-24T21:07:10.081ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
144652026-03-24T21:07:10.081ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "09377fb3-2752-481d-b375-00ec3056d7a6", "content-length": "722", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144662026-03-24T21:07:10.081ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
144672026-03-24T21:07:10.082ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c95af50-5133-4f4c-bdfb-5d7c2df01fbb", "content-length": "10", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144682026-03-24T21:07:10.082ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
144692026-03-24T21:07:10.082ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a12df6af-6a10-484e-8b35-15710beb2d4c", "content-length": "144", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144702026-03-24T21:07:10.082ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
144712026-03-24T21:07:10.082ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b377cb6c-5d8d-4b8b-b267-5edab1542956", "content-length": "147", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144722026-03-24T21:07:10.082ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
144732026-03-24T21:07:10.083ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "612a7ddd-07cd-4ca5-9ec8-48a102dba213", "content-length": "109", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144742026-03-24T21:07:10.083ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
144752026-03-24T21:07:10.083ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fcc15a8-6b30-43b2-9fc7-f321a999c063", "content-length": "109", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144762026-03-24T21:07:10.083ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
144772026-03-24T21:07:10.084ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6a2dbde-6793-400b-a2e3-3cae12aff77a", "content-length": "163", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144782026-03-24T21:07:10.084ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
144792026-03-24T21:07:10.084ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e6a1318-adf1-4ebf-bdd5-c7c2032be9b8", "content-length": "163", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144802026-03-24T21:07:10.084ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
144812026-03-24T21:07:10.085ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6395c9e9-d2c0-4dc1-88b2-e2da52654fe1", "content-length": "166", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144822026-03-24T21:07:10.085ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
144832026-03-24T21:07:10.085ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "06050447-c332-4a30-bde2-d99e16e91700", "content-length": "166", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144842026-03-24T21:07:10.085ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
144852026-03-24T21:07:10.085ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6034379-dcec-4465-aacc-13eb344357b3", "content-length": "702", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144862026-03-24T21:07:10.085ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
144872026-03-24T21:07:10.086ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "07746536-526b-4bb3-a831-9e995a21d2f4", "content-length": "727", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144882026-03-24T21:07:10.086ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
144892026-03-24T21:07:10.086ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d15f6476-05b5-409b-9b2f-4cccbab1c20b", "content-length": "729", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144902026-03-24T21:07:10.086ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
144912026-03-24T21:07:10.086ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c999d8e-0e0e-42c8-b147-676616b005db", "content-length": "728", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
144922026-03-24T21:07:10.086ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144932026-03-24T21:07:10.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144942026-03-24T21:07:10.089ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144952026-03-24T21:07:10.092ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
144962026-03-24T21:07:10.092ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144972026-03-24T21:07:10.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144982026-03-24T21:07:10.094ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144992026-03-24T21:07:10.097ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
145002026-03-24T21:07:10.097ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145012026-03-24T21:07:10.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145022026-03-24T21:07:10.103ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
145032026-03-24T21:07:10.103ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
145042026-03-24T21:07:10.171ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "228fa89b-dbc9-4c45-9753-db7429e0c816", "content-length": "110672", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145052026-03-24T21:07:10.172ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
145062026-03-24T21:07:10.172ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
145072026-03-24T21:07:10.172ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
145082026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
145092026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
145102026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
145112026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
145122026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
145132026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c70565e2-9cec-45ba-8670-42a2fd464244", "content-length": "64", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145142026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
145152026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d03c638-af4b-45d5-888b-c4ca07b39f2c", "content-length": "64", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145162026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
145172026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2509ca9-6943-4526-b395-a8091eca8b2b", "content-length": "64", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145182026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
145192026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5bf739e-8738-4397-a33b-4ff920bf9f6f", "content-length": "64", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145202026-03-24T21:07:10.172ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "612d023e-5a2f-42da-a1f3-23feaa8f4ceb", "content-length": "64", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145212026-03-24T21:07:10.173ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
145222026-03-24T21:07:10.173ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
145232026-03-24T21:07:10.180ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "7dd2eedc-f37a-47cd-8142-850317d678c6", "content-length": "407291", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145242026-03-24T21:07:10.181ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e38652d-6c53-4c9a-b2b0-61b4ee889d16", "content-length": "403233", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145252026-03-24T21:07:10.181ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "83a04b37-45bc-40b1-9ee4-ef3b8eee91fe", "content-length": "415767", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145262026-03-24T21:07:10.181ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "feec8d39-5912-4848-a8ab-cfc92ceec146", "content-length": "435107", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145272026-03-24T21:07:10.181ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6023903-3fcc-4ae1-9b80-b57cf65551de", "content-length": "406618", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145282026-03-24T21:07:10.186ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
145292026-03-24T21:07:10.186ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
145302026-03-24T21:07:10.186ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
145312026-03-24T21:07:10.186ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
145322026-03-24T21:07:10.186ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
145332026-03-24T21:07:10.188ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 (omicron_zone)
145342026-03-24T21:07:10.188ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
145352026-03-24T21:07:10.192ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "60037510-a5cb-4482-a8ae-89ce94e4d708", "content-length": "127", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145362026-03-24T21:07:10.192ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
145372026-03-24T21:07:10.196ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
145382026-03-24T21:07:10.196ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
145392026-03-24T21:07:10.196ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f00fa51b-9e9f-4219-b33a-85c28d5608af", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145402026-03-24T21:07:10.197ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
145412026-03-24T21:07:10.197ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
145422026-03-24T21:07:10.197ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
145432026-03-24T21:07:10.197ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "75a0f5ea-31db-48ee-97e8-14712f286901", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145442026-03-24T21:07:10.197ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
145452026-03-24T21:07:10.197ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
145462026-03-24T21:07:10.197ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
145472026-03-24T21:07:10.198ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6415b239-ee37-4021-bc31-0b8bbcbbd85f", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:07:10 GMT"} })
145482026-03-24T21:07:10.198ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
145492026-03-24T21:07:10.198ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
145502026-03-24T21:07:10.198ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
145512026-03-24T21:07:10.198ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145522026-03-24T21:07:10.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145532026-03-24T21:07:10.202ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145542026-03-24T21:07:10.205ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
145552026-03-24T21:07:10.205ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145562026-03-24T21:07:10.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145572026-03-24T21:07:10.326ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = da898c00-a280-40a9-aa3c-5dfd1b54176d file = nexus/db-queries/src/db/datastore/inventory.rs:1757
145582026-03-24T21:07:10.326ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = da898c00-a280-40a9-aa3c-5dfd1b54176d time_started = 2026-03-24 21:07:10.075393 UTC
145592026-03-24T21:07:10.326ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 55 reason = Dependency
145602026-03-24T21:07:10.327ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
145612026-03-24T21:07:10.327ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 762.919292ms iteration = 14 status = {"collection_id":"da898c00-a280-40a9-aa3c-5dfd1b54176d","time_done":"2026-03-24 21:07:10.198868 UTC","time_started":"2026-03-24 21:07:10.075393 UTC"}
145622026-03-24T21:07:10.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
145632026-03-24T21:07:10.329ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145642026-03-24T21:07:10.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
145652026-03-24T21:07:10.334ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
145662026-03-24T21:07:10.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
145672026-03-24T21:07:10.336ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
145682026-03-24T21:07:10.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
145692026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 26 reason = Dependency
145702026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 62.355451ms iteration = 55 status = {"Loaded":{"collection_id":"da898c00-a280-40a9-aa3c-5dfd1b54176d","time_loaded":"2026-03-24T21:07:10.326972893Z","time_started":"2026-03-24T21:07:10.075393Z"}}
145712026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 18 reason = Dependency
145722026-03-24T21:07:10.389ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
145732026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 36 reason = Dependency
145742026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 28 reason = Dependency
145752026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 16.381\u{b5}s iteration = 26 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
145762026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
145772026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
145782026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
145792026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
145802026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
145812026-03-24T21:07:10.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
145822026-03-24T21:07:10.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145832026-03-24T21:07:10.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145842026-03-24T21:07:10.391ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145852026-03-24T21:07:10.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
145862026-03-24T21:07:10.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
145872026-03-24T21:07:10.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
145882026-03-24T21:07:10.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
145892026-03-24T21:07:10.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
145902026-03-24T21:07:10.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
145912026-03-24T21:07:10.394ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
145922026-03-24T21:07:10.394ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
145932026-03-24T21:07:10.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
145942026-03-24T21:07:10.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145952026-03-24T21:07:10.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145962026-03-24T21:07:10.396ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
145972026-03-24T21:07:10.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
145982026-03-24T21:07:10.398ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145992026-03-24T21:07:10.398ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146002026-03-24T21:07:10.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146012026-03-24T21:07:10.398ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
146022026-03-24T21:07:10.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146032026-03-24T21:07:10.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146042026-03-24T21:07:10.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146052026-03-24T21:07:10.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
146062026-03-24T21:07:10.400ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
146072026-03-24T21:07:10.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
146082026-03-24T21:07:10.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146092026-03-24T21:07:10.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146102026-03-24T21:07:10.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146112026-03-24T21:07:10.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146122026-03-24T21:07:10.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146132026-03-24T21:07:10.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146142026-03-24T21:07:10.404ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146152026-03-24T21:07:10.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146162026-03-24T21:07:10.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146172026-03-24T21:07:10.405ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
146182026-03-24T21:07:10.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146192026-03-24T21:07:10.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146202026-03-24T21:07:10.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146212026-03-24T21:07:10.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146222026-03-24T21:07:10.408ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146232026-03-24T21:07:10.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146242026-03-24T21:07:10.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146252026-03-24T21:07:10.408ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.104246ms iteration = 36 status = {"physical_disks_added":0}
146262026-03-24T21:07:10.409ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
146272026-03-24T21:07:10.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146282026-03-24T21:07:10.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146292026-03-24T21:07:10.410ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
146302026-03-24T21:07:10.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146312026-03-24T21:07:10.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146322026-03-24T21:07:10.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146332026-03-24T21:07:10.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146342026-03-24T21:07:10.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146352026-03-24T21:07:10.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146362026-03-24T21:07:10.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146372026-03-24T21:07:10.414ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146382026-03-24T21:07:10.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146392026-03-24T21:07:10.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146402026-03-24T21:07:10.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146412026-03-24T21:07:10.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146422026-03-24T21:07:10.417ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146432026-03-24T21:07:10.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146442026-03-24T21:07:10.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146452026-03-24T21:07:10.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
146462026-03-24T21:07:10.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146472026-03-24T21:07:10.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146482026-03-24T21:07:10.419ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
146492026-03-24T21:07:10.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146502026-03-24T21:07:10.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146512026-03-24T21:07:10.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146522026-03-24T21:07:10.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146532026-03-24T21:07:10.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146542026-03-24T21:07:10.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146552026-03-24T21:07:10.423ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146562026-03-24T21:07:10.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146572026-03-24T21:07:10.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146582026-03-24T21:07:10.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
146592026-03-24T21:07:10.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146602026-03-24T21:07:10.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146612026-03-24T21:07:10.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146622026-03-24T21:07:10.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146632026-03-24T21:07:10.427ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146642026-03-24T21:07:10.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146652026-03-24T21:07:10.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146662026-03-24T21:07:10.428ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
146672026-03-24T21:07:10.428ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 38.998202ms iteration = 18 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"da898c00-a280-40a9-aa3c-5dfd1b54176d","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
146682026-03-24T21:07:10.429ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
146692026-03-24T21:07:10.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146702026-03-24T21:07:10.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146712026-03-24T21:07:10.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146722026-03-24T21:07:10.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146732026-03-24T21:07:10.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146742026-03-24T21:07:10.435ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
146752026-03-24T21:07:10.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146762026-03-24T21:07:10.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146772026-03-24T21:07:10.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146782026-03-24T21:07:10.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146792026-03-24T21:07:10.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146802026-03-24T21:07:10.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
146812026-03-24T21:07:10.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146822026-03-24T21:07:10.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146832026-03-24T21:07:10.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146842026-03-24T21:07:10.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146852026-03-24T21:07:10.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146862026-03-24T21:07:10.447ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
146872026-03-24T21:07:10.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146882026-03-24T21:07:10.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146892026-03-24T21:07:10.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146902026-03-24T21:07:10.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146912026-03-24T21:07:10.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146922026-03-24T21:07:10.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146932026-03-24T21:07:10.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146942026-03-24T21:07:10.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
146952026-03-24T21:07:10.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146962026-03-24T21:07:10.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146972026-03-24T21:07:10.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146982026-03-24T21:07:10.472ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
146992026-03-24T21:07:10.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147002026-03-24T21:07:10.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147012026-03-24T21:07:10.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147022026-03-24T21:07:10.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147032026-03-24T21:07:10.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147042026-03-24T21:07:10.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147052026-03-24T21:07:10.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147062026-03-24T21:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
147072026-03-24T21:07:10.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147082026-03-24T21:07:10.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147092026-03-24T21:07:10.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147102026-03-24T21:07:10.484ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
147112026-03-24T21:07:10.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147122026-03-24T21:07:10.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147132026-03-24T21:07:10.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147142026-03-24T21:07:10.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147152026-03-24T21:07:10.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147162026-03-24T21:07:10.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147172026-03-24T21:07:10.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147182026-03-24T21:07:10.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
147192026-03-24T21:07:10.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147202026-03-24T21:07:10.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147212026-03-24T21:07:10.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147222026-03-24T21:07:10.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147232026-03-24T21:07:10.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147242026-03-24T21:07:10.497ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
147252026-03-24T21:07:10.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147262026-03-24T21:07:10.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147272026-03-24T21:07:10.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147282026-03-24T21:07:10.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147292026-03-24T21:07:10.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147302026-03-24T21:07:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
147312026-03-24T21:07:10.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147322026-03-24T21:07:10.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147332026-03-24T21:07:10.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147342026-03-24T21:07:10.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147352026-03-24T21:07:10.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147362026-03-24T21:07:10.511ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
147372026-03-24T21:07:10.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147382026-03-24T21:07:10.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147392026-03-24T21:07:10.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147402026-03-24T21:07:10.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147412026-03-24T21:07:10.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147422026-03-24T21:07:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
147432026-03-24T21:07:10.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147442026-03-24T21:07:10.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147452026-03-24T21:07:10.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147462026-03-24T21:07:10.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147472026-03-24T21:07:10.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147482026-03-24T21:07:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
147492026-03-24T21:07:10.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147502026-03-24T21:07:10.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147512026-03-24T21:07:10.524ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
147522026-03-24T21:07:10.524ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
147532026-03-24T21:07:10.524ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = d5103ad9-8999-450b-a81c-0839a0b5c488 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
147542026-03-24T21:07:10.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147552026-03-24T21:07:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147562026-03-24T21:07:10.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147572026-03-24T21:07:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
147582026-03-24T21:07:10.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147592026-03-24T21:07:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147602026-03-24T21:07:10.532ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
147612026-03-24T21:07:10.532ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
147622026-03-24T21:07:10.532ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 142.927234ms iteration = 28 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
147632026-03-24T21:07:10.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 122 reason = Timeout
147642026-03-24T21:07:10.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
147652026-03-24T21:07:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
147662026-03-24T21:07:10.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147672026-03-24T21:07:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
147682026-03-24T21:07:10.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
147692026-03-24T21:07:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
147702026-03-24T21:07:10.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
147712026-03-24T21:07:10.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.32023ms iteration = 122 status = {"config_updated":false}
147722026-03-24T21:07:15.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
147732026-03-24T21:07:15.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 123 reason = Timeout
147742026-03-24T21:07:15.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147752026-03-24T21:07:15.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
147762026-03-24T21:07:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
147772026-03-24T21:07:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
147782026-03-24T21:07:15.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147792026-03-24T21:07:15.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147802026-03-24T21:07:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
147812026-03-24T21:07:15.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
147822026-03-24T21:07:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
147832026-03-24T21:07:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
147842026-03-24T21:07:15.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147852026-03-24T21:07:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
147862026-03-24T21:07:15.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
147872026-03-24T21:07:15.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.349448ms iteration = 123 status = {"config_updated":false}
147882026-03-24T21:07:15.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147892026-03-24T21:07:15.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
147902026-03-24T21:07:15.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147912026-03-24T21:07:15.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
147922026-03-24T21:07:15.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147932026-03-24T21:07:15.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
147942026-03-24T21:07:15.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
147952026-03-24T21:07:15.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
147962026-03-24T21:07:15.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
147972026-03-24T21:07:15.562ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
147982026-03-24T21:07:15.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
147992026-03-24T21:07:15.563ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
148002026-03-24T21:07:15.563ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.829291ms iteration = 62 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
148012026-03-24T21:07:15.563ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.001441ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
148022026-03-24T21:07:18.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 378 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 4b963da9-dfcd-4f5d-a908-2f9f4e02adf2 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
148032026-03-24T21:07:20.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 56 reason = Timeout
148042026-03-24T21:07:20.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 124 reason = Timeout
148052026-03-24T21:07:20.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
148062026-03-24T21:07:20.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148072026-03-24T21:07:20.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
148082026-03-24T21:07:20.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148092026-03-24T21:07:20.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148102026-03-24T21:07:20.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148112026-03-24T21:07:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
148122026-03-24T21:07:20.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148132026-03-24T21:07:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
148142026-03-24T21:07:20.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
148152026-03-24T21:07:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148162026-03-24T21:07:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
148172026-03-24T21:07:20.540ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = da898c00-a280-40a9-aa3c-5dfd1b54176d original_time_started = 2026-03-24 21:07:10.075393 UTC
148182026-03-24T21:07:20.540ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.626756ms iteration = 56 status = {"Loaded":{"collection_id":"da898c00-a280-40a9-aa3c-5dfd1b54176d","time_loaded":"2026-03-24T21:07:20.533166275Z","time_started":"2026-03-24T21:07:10.075393Z"}}
148192026-03-24T21:07:20.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
148202026-03-24T21:07:20.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.863572ms iteration = 124 status = {"config_updated":false}
148212026-03-24T21:07:20.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 42 reason = Timeout
148222026-03-24T21:07:20.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 42 reason = Timeout
148232026-03-24T21:07:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
148242026-03-24T21:07:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
148252026-03-24T21:07:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
148262026-03-24T21:07:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
148272026-03-24T21:07:20.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
148282026-03-24T21:07:20.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
148292026-03-24T21:07:20.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148302026-03-24T21:07:20.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148312026-03-24T21:07:20.586ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
148322026-03-24T21:07:20.586ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
148332026-03-24T21:07:20.586ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
148342026-03-24T21:07:20.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
148352026-03-24T21:07:20.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.064993ms iteration = 42 status = "NoSitrep"
148362026-03-24T21:07:20.588ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
148372026-03-24T21:07:20.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
148382026-03-24T21:07:20.588ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
148392026-03-24T21:07:20.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
148402026-03-24T21:07:20.590ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148412026-03-24T21:07:20.591ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
148422026-03-24T21:07:20.595ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
148432026-03-24T21:07:20.595ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
148442026-03-24T21:07:20.595ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.030661ms iteration = 42 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
148452026-03-24T21:07:25.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
148462026-03-24T21:07:25.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 125 reason = Timeout
148472026-03-24T21:07:25.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148482026-03-24T21:07:25.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148492026-03-24T21:07:25.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148502026-03-24T21:07:25.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148512026-03-24T21:07:25.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148522026-03-24T21:07:25.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148532026-03-24T21:07:25.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
148542026-03-24T21:07:25.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148552026-03-24T21:07:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
148562026-03-24T21:07:25.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148572026-03-24T21:07:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148582026-03-24T21:07:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148592026-03-24T21:07:25.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
148602026-03-24T21:07:25.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.940807ms iteration = 125 status = {"config_updated":false}
148612026-03-24T21:07:25.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148622026-03-24T21:07:25.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148632026-03-24T21:07:25.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148642026-03-24T21:07:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
148652026-03-24T21:07:25.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148662026-03-24T21:07:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148672026-03-24T21:07:25.556ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
148682026-03-24T21:07:25.556ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.000537ms iteration = 63 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
148692026-03-24T21:07:25.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
148702026-03-24T21:07:25.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
148712026-03-24T21:07:25.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
148722026-03-24T21:07:25.601ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
148732026-03-24T21:07:25.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
148742026-03-24T21:07:25.603ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.91622ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
148752026-03-24T21:07:28.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 362 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 652a7f52-df25-4855-9bd5-fe8249deaa73 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
148762026-03-24T21:07:30.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 126 reason = Timeout
148772026-03-24T21:07:30.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148782026-03-24T21:07:30.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148792026-03-24T21:07:30.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148802026-03-24T21:07:30.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
148812026-03-24T21:07:30.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148822026-03-24T21:07:30.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148832026-03-24T21:07:30.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
148842026-03-24T21:07:30.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.700291ms iteration = 126 status = {"config_updated":false}
148852026-03-24T21:07:35.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
148862026-03-24T21:07:35.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 22 reason = Timeout
148872026-03-24T21:07:35.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
148882026-03-24T21:07:35.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
148892026-03-24T21:07:35.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
148902026-03-24T21:07:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
148912026-03-24T21:07:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
148922026-03-24T21:07:35.524ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1441, tv_nsec: 738251227 } })
148932026-03-24T21:07:35.524ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
148942026-03-24T21:07:35.525ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1441, tv_nsec: 739164912 } })
148952026-03-24T21:07:35.526ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1441, tv_nsec: 740336033 } })
148962026-03-24T21:07:35.527ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
148972026-03-24T21:07:35.527ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.504335ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
148982026-03-24T21:07:35.527ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
148992026-03-24T21:07:35.527ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "835eac4c-03da-440c-8203-a3c351d82231", "content-length": "26", "date": "Tue, 24 Mar 2026 21:07:35 GMT"} })
149002026-03-24T21:07:35.527ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
149012026-03-24T21:07:35.527ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
149022026-03-24T21:07:35.527ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1441, tv_nsec: 741676619 } })
149032026-03-24T21:07:35.528ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
149042026-03-24T21:07:35.528ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 8.368806ms iteration = 22 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
149052026-03-24T21:07:35.529ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 9.228799ms iteration = 22 status = {}
149062026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
149072026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
149082026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
149092026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
149102026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 33 reason = Timeout
149112026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
149122026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149132026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
149142026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
149152026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
149162026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 37 reason = Timeout
149172026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 57 reason = Timeout
149182026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
149192026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 127 reason = Timeout
149202026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
149212026-03-24T21:07:35.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
149222026-03-24T21:07:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
149232026-03-24T21:07:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
149242026-03-24T21:07:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
149252026-03-24T21:07:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
149262026-03-24T21:07:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
149272026-03-24T21:07:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149282026-03-24T21:07:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
149292026-03-24T21:07:35.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
149302026-03-24T21:07:35.535ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149312026-03-24T21:07:35.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149322026-03-24T21:07:35.536ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149332026-03-24T21:07:35.536ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149342026-03-24T21:07:35.538ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149352026-03-24T21:07:35.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
149362026-03-24T21:07:35.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149372026-03-24T21:07:35.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
149382026-03-24T21:07:35.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
149392026-03-24T21:07:35.539ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
149402026-03-24T21:07:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
149412026-03-24T21:07:35.540ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149422026-03-24T21:07:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
149432026-03-24T21:07:35.542ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
149442026-03-24T21:07:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
149452026-03-24T21:07:35.542ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
149462026-03-24T21:07:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
149472026-03-24T21:07:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
149482026-03-24T21:07:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
149492026-03-24T21:07:35.543ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
149502026-03-24T21:07:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
149512026-03-24T21:07:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
149522026-03-24T21:07:35.543ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
149532026-03-24T21:07:35.543ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
149542026-03-24T21:07:35.544ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
149552026-03-24T21:07:35.544ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 11.13049ms iteration = 127 status = {"config_updated":false}
149562026-03-24T21:07:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
149572026-03-24T21:07:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
149582026-03-24T21:07:35.544ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
149592026-03-24T21:07:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
149602026-03-24T21:07:35.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149612026-03-24T21:07:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
149622026-03-24T21:07:35.545ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
149632026-03-24T21:07:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149642026-03-24T21:07:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
149652026-03-24T21:07:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
149662026-03-24T21:07:35.546ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
149672026-03-24T21:07:35.546ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
149682026-03-24T21:07:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
149692026-03-24T21:07:35.547ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149702026-03-24T21:07:35.548ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = da898c00-a280-40a9-aa3c-5dfd1b54176d original_time_started = 2026-03-24 21:07:10.075393 UTC
149712026-03-24T21:07:35.548ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 15.667312ms iteration = 57 status = {"Loaded":{"collection_id":"da898c00-a280-40a9-aa3c-5dfd1b54176d","time_loaded":"2026-03-24T21:07:35.532930787Z","time_started":"2026-03-24T21:07:10.075393Z"}}
149722026-03-24T21:07:35.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149732026-03-24T21:07:35.549ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 16.231217ms iteration = 22 status = 0
149742026-03-24T21:07:35.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149752026-03-24T21:07:35.549ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1441, tv_nsec: 763311233 } })
149762026-03-24T21:07:35.549ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149772026-03-24T21:07:35.549ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
149782026-03-24T21:07:35.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
149792026-03-24T21:07:35.550ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
149802026-03-24T21:07:35.550ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1441, tv_nsec: 764105144 } })
149812026-03-24T21:07:35.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
149822026-03-24T21:07:35.551ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
149832026-03-24T21:07:35.551ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
149842026-03-24T21:07:35.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149852026-03-24T21:07:35.551ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "23516553-6945-4fe2-af90-df7a076c0b35", "content-length": "26", "date": "Tue, 24 Mar 2026 21:07:35 GMT"} })
149862026-03-24T21:07:35.551ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
149872026-03-24T21:07:35.551ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
149882026-03-24T21:07:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
149892026-03-24T21:07:35.552ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
149902026-03-24T21:07:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
149912026-03-24T21:07:35.553ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1441, tv_nsec: 767151466 } })
149922026-03-24T21:07:35.554ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
149932026-03-24T21:07:35.554ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 21.256943ms iteration = 33 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
149942026-03-24T21:07:35.555ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
149952026-03-24T21:07:35.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149962026-03-24T21:07:35.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149972026-03-24T21:07:35.556ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1441, tv_nsec: 769875050 } })
149982026-03-24T21:07:35.557ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
149992026-03-24T21:07:35.557ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
150002026-03-24T21:07:35.557ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
150012026-03-24T21:07:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
150022026-03-24T21:07:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
150032026-03-24T21:07:35.559ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
150042026-03-24T21:07:35.559ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
150052026-03-24T21:07:35.559ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
150062026-03-24T21:07:35.559ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
150072026-03-24T21:07:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
150082026-03-24T21:07:35.559ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
150092026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
150102026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
150112026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
150122026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
150132026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
150142026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
150152026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
150162026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
150172026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
150182026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
150192026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 33 reason = Timeout
150202026-03-24T21:07:35.560ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
150212026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150222026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
150232026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
150242026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
150252026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
150262026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
150272026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150282026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
150292026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
150302026-03-24T21:07:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
150312026-03-24T21:07:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
150322026-03-24T21:07:35.561ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
150332026-03-24T21:07:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
150342026-03-24T21:07:35.566ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150352026-03-24T21:07:35.566ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
150362026-03-24T21:07:35.566ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150372026-03-24T21:07:35.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
150382026-03-24T21:07:35.567ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
150392026-03-24T21:07:35.567ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
150402026-03-24T21:07:35.567ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
150412026-03-24T21:07:35.567ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 7.493902ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
150422026-03-24T21:07:35.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
150432026-03-24T21:07:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
150442026-03-24T21:07:35.569ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
150452026-03-24T21:07:35.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
150462026-03-24T21:07:35.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
150472026-03-24T21:07:35.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150482026-03-24T21:07:35.570ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 12.670261ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
150492026-03-24T21:07:35.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150502026-03-24T21:07:35.570ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 37.766948ms iteration = 37 status = {"physical_disks_added":0}
150512026-03-24T21:07:35.571ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
150522026-03-24T21:07:35.571ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
150532026-03-24T21:07:35.571ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
150542026-03-24T21:07:35.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
150552026-03-24T21:07:35.574ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
150562026-03-24T21:07:35.574ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
150572026-03-24T21:07:35.574ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 13.981487ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
150582026-03-24T21:07:35.577ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
150592026-03-24T21:07:35.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
150602026-03-24T21:07:35.581ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150612026-03-24T21:07:35.582ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 22 reason = Timeout
150622026-03-24T21:07:35.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 43 reason = Timeout
150632026-03-24T21:07:35.582ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 19.061\u{b5}s iteration = 22 status = {"disabled":true,"errors":[],"sps":[]}
150642026-03-24T21:07:35.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 43 reason = Timeout
150652026-03-24T21:07:35.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
150662026-03-24T21:07:35.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
150672026-03-24T21:07:35.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
150682026-03-24T21:07:35.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
150692026-03-24T21:07:35.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
150702026-03-24T21:07:35.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
150712026-03-24T21:07:35.584ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
150722026-03-24T21:07:35.584ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
150732026-03-24T21:07:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
150742026-03-24T21:07:35.585ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150752026-03-24T21:07:35.585ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150762026-03-24T21:07:35.586ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150772026-03-24T21:07:35.587ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
150782026-03-24T21:07:35.587ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
150792026-03-24T21:07:35.587ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
150802026-03-24T21:07:35.587ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 54.404016ms iteration = 64 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
150812026-03-24T21:07:35.588ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
150822026-03-24T21:07:35.588ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
150832026-03-24T21:07:35.588ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
150842026-03-24T21:07:35.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
150852026-03-24T21:07:35.589ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
150862026-03-24T21:07:35.589ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
150872026-03-24T21:07:35.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
150882026-03-24T21:07:35.590ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
150892026-03-24T21:07:35.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
150902026-03-24T21:07:35.591ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 8.986782ms iteration = 43 status = "NoSitrep"
150912026-03-24T21:07:35.591ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
150922026-03-24T21:07:35.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
150932026-03-24T21:07:35.593ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150942026-03-24T21:07:35.594ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
150952026-03-24T21:07:35.595ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
150962026-03-24T21:07:35.595ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "304718e9-12dc-4b3b-b35a-b304ae718e36", "content-length": "691", "date": "Tue, 24 Mar 2026 21:07:35 GMT"} })
150972026-03-24T21:07:35.595ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
150982026-03-24T21:07:35.595ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
150992026-03-24T21:07:35.595ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
151002026-03-24T21:07:35.595ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 62.66085ms iteration = 22 status = {}
151012026-03-24T21:07:35.595ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
151022026-03-24T21:07:35.595ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
151032026-03-24T21:07:35.595ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
151042026-03-24T21:07:35.595ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
151052026-03-24T21:07:35.595ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
151062026-03-24T21:07:35.595ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151072026-03-24T21:07:35.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151082026-03-24T21:07:35.599ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
151092026-03-24T21:07:35.599ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
151102026-03-24T21:07:35.599ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
151112026-03-24T21:07:35.600ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
151122026-03-24T21:07:35.600ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 17.907073ms iteration = 43 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
151132026-03-24T21:07:35.632ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
151142026-03-24T21:07:35.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
151152026-03-24T21:07:35.637ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
151162026-03-24T21:07:35.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
151172026-03-24T21:07:35.640ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151182026-03-24T21:07:35.642ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
151192026-03-24T21:07:35.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151202026-03-24T21:07:35.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151212026-03-24T21:07:35.646ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 86.919754ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
151222026-03-24T21:07:35.646ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
151232026-03-24T21:07:35.646ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 86.941955ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
151242026-03-24T21:07:35.648ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 88.032274ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
151252026-03-24T21:07:35.651ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
151262026-03-24T21:07:35.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
151272026-03-24T21:07:35.655ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 94.939041ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
151282026-03-24T21:07:35.692ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 146.104129ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
151292026-03-24T21:07:35.693ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
151302026-03-24T21:07:35.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
151312026-03-24T21:07:35.740ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 180.505617ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
151322026-03-24T21:07:35.754ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
151332026-03-24T21:07:35.754ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
151342026-03-24T21:07:35.762ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "19b9b563-daa8-4436-8bbb-85890986ffe8", "content-length": "372", "date": "Tue, 24 Mar 2026 21:07:35 GMT"} })
151352026-03-24T21:07:35.762ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151362026-03-24T21:07:35.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151372026-03-24T21:07:35.815ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
151382026-03-24T21:07:35.815ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
151392026-03-24T21:07:35.815ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9ef9ac7d-d05f-417f-bf51-a05cf72f3258", "date": "Tue, 24 Mar 2026 21:07:35 GMT"} })
151402026-03-24T21:07:35.815ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151412026-03-24T21:07:35.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151422026-03-24T21:07:35.818ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151432026-03-24T21:07:35.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151442026-03-24T21:07:35.821ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151452026-03-24T21:07:35.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151462026-03-24T21:07:35.824ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
151472026-03-24T21:07:35.824ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
151482026-03-24T21:07:35.826ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "459ca731-81c6-41f8-a1c5-2d88db7cf8f3", "date": "Tue, 24 Mar 2026 21:07:35 GMT"} })
151492026-03-24T21:07:35.826ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 265.711814ms iteration = 33 status = {}
151502026-03-24T21:07:35.865ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "0160457f-38f9-404c-ad8c-079a06277bcf", "date": "Tue, 24 Mar 2026 21:07:35 GMT"} })
151512026-03-24T21:07:35.865ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
151522026-03-24T21:07:35.865ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151532026-03-24T21:07:35.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151542026-03-24T21:07:35.911ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151552026-03-24T21:07:35.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151562026-03-24T21:07:35.916ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151572026-03-24T21:07:35.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151582026-03-24T21:07:35.917ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151592026-03-24T21:07:35.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151602026-03-24T21:07:35.920ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 387.207341ms iteration = 22 status = {}
151612026-03-24T21:07:38.551ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 205fe2fe-90d0-4b13-97c5-12a59eb35710 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
151622026-03-24T21:07:38.551ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 205fe2fe-90d0-4b13-97c5-12a59eb35710 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
151632026-03-24T21:07:38.551ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 205fe2fe-90d0-4b13-97c5-12a59eb35710 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
151642026-03-24T21:07:38.554ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2880 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = 205fe2fe-90d0-4b13-97c5-12a59eb35710 response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
151652026-03-24T21:07:38.554ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = f38047b4-1963-4c3a-8d19-fece12bb454f uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
151662026-03-24T21:07:38.554ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = f38047b4-1963-4c3a-8d19-fece12bb454f roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
151672026-03-24T21:07:38.554ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = f38047b4-1963-4c3a-8d19-fece12bb454f resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
151682026-03-24T21:07:38.556ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2405 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:35434 req_id = f38047b4-1963-4c3a-8d19-fece12bb454f response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
151692026-03-24T21:07:38.557ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 376 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = cf4c6623-e0ec-46b0-9dd7-2aaf4edb8264 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
151702026-03-24T21:07:40.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 128 reason = Timeout
151712026-03-24T21:07:40.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
151722026-03-24T21:07:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
151732026-03-24T21:07:40.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151742026-03-24T21:07:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
151752026-03-24T21:07:40.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
151762026-03-24T21:07:40.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
151772026-03-24T21:07:40.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
151782026-03-24T21:07:40.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.169103ms iteration = 128 status = {"config_updated":false}
151792026-03-24T21:07:45.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
151802026-03-24T21:07:45.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 129 reason = Timeout
151812026-03-24T21:07:45.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
151822026-03-24T21:07:45.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
151832026-03-24T21:07:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
151842026-03-24T21:07:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
151852026-03-24T21:07:45.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151862026-03-24T21:07:45.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151872026-03-24T21:07:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
151882026-03-24T21:07:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
151892026-03-24T21:07:45.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
151902026-03-24T21:07:45.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
151912026-03-24T21:07:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
151922026-03-24T21:07:45.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
151932026-03-24T21:07:45.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
151942026-03-24T21:07:45.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.561657ms iteration = 129 status = {"config_updated":false}
151952026-03-24T21:07:45.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
151962026-03-24T21:07:45.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
151972026-03-24T21:07:45.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151982026-03-24T21:07:45.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
151992026-03-24T21:07:45.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
152002026-03-24T21:07:45.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
152012026-03-24T21:07:45.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
152022026-03-24T21:07:45.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
152032026-03-24T21:07:45.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
152042026-03-24T21:07:45.562ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
152052026-03-24T21:07:45.562ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.208158ms iteration = 65 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
152062026-03-24T21:07:45.601ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
152072026-03-24T21:07:45.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
152082026-03-24T21:07:45.602ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.150837ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
152092026-03-24T21:07:48.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 437 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 979f2019-90a3-401b-a862-a10e01a8dcc7 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
152102026-03-24T21:07:50.532ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 58 reason = Timeout
152112026-03-24T21:07:50.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 130 reason = Timeout
152122026-03-24T21:07:50.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
152132026-03-24T21:07:50.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152142026-03-24T21:07:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
152152026-03-24T21:07:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152162026-03-24T21:07:50.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152172026-03-24T21:07:50.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152182026-03-24T21:07:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
152192026-03-24T21:07:50.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152202026-03-24T21:07:50.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152212026-03-24T21:07:50.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
152222026-03-24T21:07:50.540ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
152232026-03-24T21:07:50.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
152242026-03-24T21:07:50.542ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
152252026-03-24T21:07:50.542ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = da898c00-a280-40a9-aa3c-5dfd1b54176d original_time_started = 2026-03-24 21:07:10.075393 UTC
152262026-03-24T21:07:50.542ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 10.187855ms iteration = 130 status = {"config_updated":false}
152272026-03-24T21:07:50.542ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 10.233136ms iteration = 58 status = {"Loaded":{"collection_id":"da898c00-a280-40a9-aa3c-5dfd1b54176d","time_loaded":"2026-03-24T21:07:50.532684729Z","time_started":"2026-03-24T21:07:10.075393Z"}}
152282026-03-24T21:07:50.581ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 44 reason = Timeout
152292026-03-24T21:07:50.581ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
152302026-03-24T21:07:50.581ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 44 reason = Timeout
152312026-03-24T21:07:50.581ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
152322026-03-24T21:07:50.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
152332026-03-24T21:07:50.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
152342026-03-24T21:07:50.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
152352026-03-24T21:07:50.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
152362026-03-24T21:07:50.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152372026-03-24T21:07:50.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152382026-03-24T21:07:50.586ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
152392026-03-24T21:07:50.586ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
152402026-03-24T21:07:50.586ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
152412026-03-24T21:07:50.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
152422026-03-24T21:07:50.589ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.511062ms iteration = 44 status = "NoSitrep"
152432026-03-24T21:07:50.590ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
152442026-03-24T21:07:50.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
152452026-03-24T21:07:50.591ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
152462026-03-24T21:07:50.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
152472026-03-24T21:07:50.592ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152482026-03-24T21:07:50.594ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
152492026-03-24T21:07:50.598ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
152502026-03-24T21:07:50.598ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
152512026-03-24T21:07:50.598ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 16.775812ms iteration = 44 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
152522026-03-24T21:07:55.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
152532026-03-24T21:07:55.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 131 reason = Timeout
152542026-03-24T21:07:55.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
152552026-03-24T21:07:55.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152562026-03-24T21:07:55.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152572026-03-24T21:07:55.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
152582026-03-24T21:07:55.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152592026-03-24T21:07:55.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152602026-03-24T21:07:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
152612026-03-24T21:07:55.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152622026-03-24T21:07:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152632026-03-24T21:07:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
152642026-03-24T21:07:55.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
152652026-03-24T21:07:55.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
152662026-03-24T21:07:55.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
152672026-03-24T21:07:55.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.978098ms iteration = 131 status = {"config_updated":false}
152682026-03-24T21:07:55.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
152692026-03-24T21:07:55.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
152702026-03-24T21:07:55.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152712026-03-24T21:07:55.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
152722026-03-24T21:07:55.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
152732026-03-24T21:07:55.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
152742026-03-24T21:07:55.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
152752026-03-24T21:07:55.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
152762026-03-24T21:07:55.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
152772026-03-24T21:07:55.561ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
152782026-03-24T21:07:55.561ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.431007ms iteration = 66 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
152792026-03-24T21:07:55.600ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
152802026-03-24T21:07:55.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
152812026-03-24T21:07:55.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 83.941783ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
152822026-03-24T21:07:58.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 422 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 675f202c-dcff-4828-92b1-5f0eec6b7f7f response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
152832026-03-24T21:08:00.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 132 reason = Timeout
152842026-03-24T21:08:00.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152852026-03-24T21:08:00.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152862026-03-24T21:08:00.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152872026-03-24T21:08:00.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
152882026-03-24T21:08:00.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152892026-03-24T21:08:00.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152902026-03-24T21:08:00.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
152912026-03-24T21:08:00.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.466114ms iteration = 132 status = {"config_updated":false}
152922026-03-24T21:08:05.515ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
152932026-03-24T21:08:05.515ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
152942026-03-24T21:08:05.515ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
152952026-03-24T21:08:05.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
152962026-03-24T21:08:05.516ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
152972026-03-24T21:08:05.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
152982026-03-24T21:08:05.519ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152992026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
153002026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
153012026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
153022026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
153032026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
153042026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
153052026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
153062026-03-24T21:08:05.520ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2026-03-24 20:58:05.520497982 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
153072026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
153082026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
153092026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 23 reason = Timeout
153102026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
153112026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
153122026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
153132026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
153142026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
153152026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
153162026-03-24T21:08:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
153172026-03-24T21:08:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
153182026-03-24T21:08:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
153192026-03-24T21:08:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
153202026-03-24T21:08:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
153212026-03-24T21:08:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("27f3a9c5-67c7-41c5-994d-9db513cda557.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2f7ea2ed-fd45-4cd7-9fb1-f48119541118.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 735083943 } })
153222026-03-24T21:08:05.521ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("93b6e989-1f7d-4884-aec9-ecbc076ed6c4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 735093983 } })
153232026-03-24T21:08:05.521ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
153242026-03-24T21:08:05.521ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
153252026-03-24T21:08:05.521ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
153262026-03-24T21:08:05.521ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = ExternalNetworking step id = Ensure
153272026-03-24T21:08:05.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153282026-03-24T21:08:05.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153292026-03-24T21:08:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
153302026-03-24T21:08:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.630584ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
153312026-03-24T21:08:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
153322026-03-24T21:08:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.581503ms iteration = 12 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
153332026-03-24T21:08:05.522ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
153342026-03-24T21:08:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
153352026-03-24T21:08:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
153362026-03-24T21:08:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
153372026-03-24T21:08:05.524ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2026-03-24 20:58:05.520497982 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
153382026-03-24T21:08:05.524ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.565946ms iteration = 12 status = {"expiration":"2026-03-24T20:58:05.520497982Z","pruned":[]}
153392026-03-24T21:08:05.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
153402026-03-24T21:08:05.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
153412026-03-24T21:08:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "a7814dcc-bbdd-4ddf-8f19-7d398866cd23", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
153422026-03-24T21:08:05.525ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
153432026-03-24T21:08:05.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
153442026-03-24T21:08:05.525ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 739113561 } })
153452026-03-24T21:08:05.525ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 739238995 } })
153462026-03-24T21:08:05.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153472026-03-24T21:08:05.525ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "5694764a-a629-46d1-8c55-7d22eaae2940", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
153482026-03-24T21:08:05.525ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
153492026-03-24T21:08:05.525ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
153502026-03-24T21:08:05.525ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 4.958494ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
153512026-03-24T21:08:05.525ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
153522026-03-24T21:08:05.525ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
153532026-03-24T21:08:05.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153542026-03-24T21:08:05.526ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 739940854 } })
153552026-03-24T21:08:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "024da57f-bf11-4a1e-b3b3-3e1e68325d4d", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
153562026-03-24T21:08:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
153572026-03-24T21:08:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d8bbc36e-9995-488f-a276-a883d7897c73", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
153582026-03-24T21:08:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
153592026-03-24T21:08:05.526ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "08fe610b-d4ad-4451-8a8b-9848d72328df", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
153602026-03-24T21:08:05.526ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
153612026-03-24T21:08:05.526ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
153622026-03-24T21:08:05.526ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 6.054343ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
153632026-03-24T21:08:05.527ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
153642026-03-24T21:08:05.527ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
153652026-03-24T21:08:05.527ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
153662026-03-24T21:08:05.527ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 7.41807ms iteration = 23 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
153672026-03-24T21:08:05.528ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.834191ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
153682026-03-24T21:08:05.528ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "53eb0dd5-032a-4343-80b3-ccfd7e43739b", "content-length": "26", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} })
153692026-03-24T21:08:05.528ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
153702026-03-24T21:08:05.528ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
153712026-03-24T21:08:05.528ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 742541904 } })
153722026-03-24T21:08:05.530ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 10.236646ms iteration = 23 status = {}
153732026-03-24T21:08:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153742026-03-24T21:08:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153752026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
153762026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
153772026-03-24T21:08:05.533ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
153782026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
153792026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
153802026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 59 reason = Timeout
153812026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
153822026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
153832026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 133 reason = Timeout
153842026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
153852026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
153862026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
153872026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
153882026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 29 reason = Timeout
153892026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
153902026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 34 reason = Timeout
153912026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
153922026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 38 reason = Timeout
153932026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
153942026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
153952026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
153962026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
153972026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
153982026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
153992026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
154002026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
154012026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
154022026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
154032026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
154042026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
154052026-03-24T21:08:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
154062026-03-24T21:08:05.535ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154072026-03-24T21:08:05.536ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154082026-03-24T21:08:05.538ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
154092026-03-24T21:08:05.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
154102026-03-24T21:08:05.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
154112026-03-24T21:08:05.539ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
154122026-03-24T21:08:05.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
154132026-03-24T21:08:05.541ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154142026-03-24T21:08:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
154152026-03-24T21:08:05.542ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
154162026-03-24T21:08:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
154172026-03-24T21:08:05.543ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
154182026-03-24T21:08:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
154192026-03-24T21:08:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
154202026-03-24T21:08:05.545ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
154212026-03-24T21:08:05.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
154222026-03-24T21:08:05.547ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
154232026-03-24T21:08:05.547ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
154242026-03-24T21:08:05.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
154252026-03-24T21:08:05.548ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 15.489028ms iteration = 23 status = 0
154262026-03-24T21:08:05.550ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154272026-03-24T21:08:05.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
154282026-03-24T21:08:05.553ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
154292026-03-24T21:08:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
154302026-03-24T21:08:05.555ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
154312026-03-24T21:08:05.555ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
154322026-03-24T21:08:05.555ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
154332026-03-24T21:08:05.555ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
154342026-03-24T21:08:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
154352026-03-24T21:08:05.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
154362026-03-24T21:08:05.557ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = da898c00-a280-40a9-aa3c-5dfd1b54176d original_time_started = 2026-03-24 21:07:10.075393 UTC
154372026-03-24T21:08:05.557ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 24.176252ms iteration = 59 status = {"Loaded":{"collection_id":"da898c00-a280-40a9-aa3c-5dfd1b54176d","time_loaded":"2026-03-24T21:08:05.533036730Z","time_started":"2026-03-24T21:07:10.075393Z"}}
154382026-03-24T21:08:05.558ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 27 reason = Timeout
154392026-03-24T21:08:05.558ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
154402026-03-24T21:08:05.558ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 12 reason = Timeout
154412026-03-24T21:08:05.558ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
154422026-03-24T21:08:05.558ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 20.28\u{b5}s iteration = 27 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
154432026-03-24T21:08:05.558ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
154442026-03-24T21:08:05.558ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
154452026-03-24T21:08:05.558ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 12 reason = Timeout
154462026-03-24T21:08:05.558ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
154472026-03-24T21:08:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
154482026-03-24T21:08:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
154492026-03-24T21:08:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
154502026-03-24T21:08:05.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
154512026-03-24T21:08:05.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
154522026-03-24T21:08:05.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
154532026-03-24T21:08:05.561ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
154542026-03-24T21:08:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
154552026-03-24T21:08:05.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
154562026-03-24T21:08:05.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
154572026-03-24T21:08:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
154582026-03-24T21:08:05.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
154592026-03-24T21:08:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
154602026-03-24T21:08:05.561ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 34 reason = Timeout
154612026-03-24T21:08:05.561ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
154622026-03-24T21:08:05.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
154632026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
154642026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
154652026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
154662026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
154672026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
154682026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
154692026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
154702026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
154712026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
154722026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
154732026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
154742026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
154752026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
154762026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
154772026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
154782026-03-24T21:08:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
154792026-03-24T21:08:05.563ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
154802026-03-24T21:08:05.563ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 5.651922ms iteration = 12 status = {"by_rx":{},"error":null}
154812026-03-24T21:08:05.565ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 10.451701ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
154822026-03-24T21:08:05.567ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 7.689437ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
154832026-03-24T21:08:05.573ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
154842026-03-24T21:08:05.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
154852026-03-24T21:08:05.576ZDEBGnexus (BackgroundTasks): activating background_task = trust_quorum_manager iteration = 12 reason = Timeout
154862026-03-24T21:08:05.576ZDEBGnexus (BackgroundTasks): activating background_task = attached_subnet_manager iteration = 12 reason = Timeout
154872026-03-24T21:08:05.576ZINFOnexus (BackgroundTasks): starting attached subnet manager work background_task = attached_subnet_manager file = nexus/src/app/background/tasks/attached_subnets.rs:374
154882026-03-24T21:08:05.576ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
154892026-03-24T21:08:05.576ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 12 reason = Timeout
154902026-03-24T21:08:05.576ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
154912026-03-24T21:08:05.576ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
154922026-03-24T21:08:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
154932026-03-24T21:08:05.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
154942026-03-24T21:08:05.580ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 794218237 } })
154952026-03-24T21:08:05.582ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 23 reason = Timeout
154962026-03-24T21:08:05.582ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 14.23\u{b5}s iteration = 23 status = {"disabled":true,"errors":[],"sps":[]}
154972026-03-24T21:08:05.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 45 reason = Timeout
154982026-03-24T21:08:05.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 45 reason = Timeout
154992026-03-24T21:08:05.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
155002026-03-24T21:08:05.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
155012026-03-24T21:08:05.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
155022026-03-24T21:08:05.583ZDEBGnexus (BackgroundTasks): roles background_task = attached_subnet_manager roles = RoleSet { roles: {} }
155032026-03-24T21:08:05.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
155042026-03-24T21:08:05.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = attached_subnet_manager resource = Database result = Ok(())
155052026-03-24T21:08:05.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155062026-03-24T21:08:05.619ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155072026-03-24T21:08:05.622ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
155082026-03-24T21:08:05.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
155092026-03-24T21:08:05.622ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
155102026-03-24T21:08:05.622ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
155112026-03-24T21:08:05.622ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
155122026-03-24T21:08:05.622ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
155132026-03-24T21:08:05.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
155142026-03-24T21:08:05.623ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
155152026-03-24T21:08:05.623ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
155162026-03-24T21:08:05.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
155172026-03-24T21:08:05.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
155182026-03-24T21:08:05.632ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155192026-03-24T21:08:05.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155202026-03-24T21:08:05.635ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155212026-03-24T21:08:05.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
155222026-03-24T21:08:05.636ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
155232026-03-24T21:08:05.637ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
155242026-03-24T21:08:05.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
155252026-03-24T21:08:05.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
155262026-03-24T21:08:05.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
155272026-03-24T21:08:05.638ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 80.257274ms iteration = 12 status = {"dispatched":[],"errors":[],"glob_version":"242.0.0","globs_reprocessed":{},"no_receivers":[]}
155282026-03-24T21:08:05.638ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155292026-03-24T21:08:05.640ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
155302026-03-24T21:08:05.640ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
155312026-03-24T21:08:05.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155322026-03-24T21:08:05.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
155332026-03-24T21:08:05.640ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155342026-03-24T21:08:05.641ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155352026-03-24T21:08:05.641ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
155362026-03-24T21:08:05.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
155372026-03-24T21:08:05.643ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
155382026-03-24T21:08:05.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
155392026-03-24T21:08:05.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
155402026-03-24T21:08:05.644ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
155412026-03-24T21:08:05.644ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
155422026-03-24T21:08:05.645ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Fleet result = Ok(())
155432026-03-24T21:08:05.645ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
155442026-03-24T21:08:05.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
155452026-03-24T21:08:05.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
155462026-03-24T21:08:05.645ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155472026-03-24T21:08:05.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
155482026-03-24T21:08:05.646ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155492026-03-24T21:08:05.648ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
155502026-03-24T21:08:05.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
155512026-03-24T21:08:05.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
155522026-03-24T21:08:05.649ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155532026-03-24T21:08:05.653ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
155542026-03-24T21:08:05.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
155552026-03-24T21:08:05.654ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155562026-03-24T21:08:05.654ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
155572026-03-24T21:08:05.655ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155582026-03-24T21:08:05.656ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
155592026-03-24T21:08:05.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
155602026-03-24T21:08:05.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
155612026-03-24T21:08:05.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155622026-03-24T21:08:05.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
155632026-03-24T21:08:05.658ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
155642026-03-24T21:08:05.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
155652026-03-24T21:08:05.658ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
155662026-03-24T21:08:05.659ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155672026-03-24T21:08:05.659ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 873671780 } })
155682026-03-24T21:08:05.660ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
155692026-03-24T21:08:05.660ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155702026-03-24T21:08:05.660ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155712026-03-24T21:08:05.660ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 874650956 } })
155722026-03-24T21:08:05.661ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
155732026-03-24T21:08:05.661ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155742026-03-24T21:08:05.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
155752026-03-24T21:08:05.661ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
155762026-03-24T21:08:05.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
155772026-03-24T21:08:05.662ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
155782026-03-24T21:08:05.662ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
155792026-03-24T21:08:05.662ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
155802026-03-24T21:08:05.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
155812026-03-24T21:08:05.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
155822026-03-24T21:08:05.663ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "34c3e184-f714-4aea-95e0-079ba3ce2546", "content-length": "26", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} })
155832026-03-24T21:08:05.663ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
155842026-03-24T21:08:05.663ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
155852026-03-24T21:08:05.663ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
155862026-03-24T21:08:05.663ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
155872026-03-24T21:08:05.663ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 81.21394ms iteration = 45 status = "NoSitrep"
155882026-03-24T21:08:05.664ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155892026-03-24T21:08:05.664ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
155902026-03-24T21:08:05.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
155912026-03-24T21:08:05.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
155922026-03-24T21:08:05.664ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
155932026-03-24T21:08:05.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
155942026-03-24T21:08:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
155952026-03-24T21:08:05.665ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
155962026-03-24T21:08:05.665ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 879424035 } })
155972026-03-24T21:08:05.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155982026-03-24T21:08:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
155992026-03-24T21:08:05.666ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
156002026-03-24T21:08:05.666ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
156012026-03-24T21:08:05.666ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
156022026-03-24T21:08:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
156032026-03-24T21:08:05.668ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1471, tv_nsec: 882166089 } })
156042026-03-24T21:08:05.668ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
156052026-03-24T21:08:05.669ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
156062026-03-24T21:08:05.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
156072026-03-24T21:08:05.669ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
156082026-03-24T21:08:05.669ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
156092026-03-24T21:08:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
156102026-03-24T21:08:05.670ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
156112026-03-24T21:08:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
156122026-03-24T21:08:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
156132026-03-24T21:08:05.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
156142026-03-24T21:08:05.671ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 113.262554ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
156152026-03-24T21:08:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
156162026-03-24T21:08:05.671ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
156172026-03-24T21:08:05.673ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
156182026-03-24T21:08:05.674ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
156192026-03-24T21:08:05.674ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
156202026-03-24T21:08:05.674ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 119.653996ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
156212026-03-24T21:08:05.676ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
156222026-03-24T21:08:05.678ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
156232026-03-24T21:08:05.678ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 145.149534ms iteration = 133 status = {"config_updated":false}
156242026-03-24T21:08:05.679ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
156252026-03-24T21:08:05.680ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
156262026-03-24T21:08:05.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
156272026-03-24T21:08:05.681ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
156282026-03-24T21:08:05.684ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 123.828159ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
156292026-03-24T21:08:05.689ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
156302026-03-24T21:08:05.692ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
156312026-03-24T21:08:05.692ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = SledList step id = Fetch
156322026-03-24T21:08:05.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
156332026-03-24T21:08:05.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
156342026-03-24T21:08:05.694ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
156352026-03-24T21:08:05.694ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
156362026-03-24T21:08:05.694ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
156372026-03-24T21:08:05.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
156382026-03-24T21:08:05.695ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
156392026-03-24T21:08:05.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
156402026-03-24T21:08:05.714ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
156412026-03-24T21:08:05.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
156422026-03-24T21:08:05.717ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
156432026-03-24T21:08:05.717ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
156442026-03-24T21:08:05.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
156452026-03-24T21:08:05.729ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
156462026-03-24T21:08:05.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
156472026-03-24T21:08:05.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156482026-03-24T21:08:05.733ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156492026-03-24T21:08:05.735ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
156502026-03-24T21:08:05.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
156512026-03-24T21:08:05.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
156522026-03-24T21:08:05.736ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
156532026-03-24T21:08:05.737ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
156542026-03-24T21:08:05.737ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
156552026-03-24T21:08:05.737ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
156562026-03-24T21:08:05.737ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
156572026-03-24T21:08:05.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
156582026-03-24T21:08:05.739ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 179.353596ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
156592026-03-24T21:08:05.741ZINFOnexus (BackgroundTasks): Loaded 0 active trust quorum configurations from database background_task = trust_quorum_manager bgtask = TrustQuorumManager file = nexus/src/app/background/tasks/trust_quorum.rs:83
156602026-03-24T21:08:05.741ZDEBGnexus (BackgroundTasks): activation complete background_task = trust_quorum_manager elapsed = 165.590606ms iteration = 12 status = {"PerRackStatus":{"errors":[],"statuses":[]}}
156612026-03-24T21:08:05.742ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
156622026-03-24T21:08:05.742ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
156632026-03-24T21:08:05.742ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 159.375938ms iteration = 45 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
156642026-03-24T21:08:05.742ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
156652026-03-24T21:08:05.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
156662026-03-24T21:08:05.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156672026-03-24T21:08:05.744ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156682026-03-24T21:08:05.748ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
156692026-03-24T21:08:05.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
156702026-03-24T21:08:05.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
156712026-03-24T21:08:05.750ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
156722026-03-24T21:08:05.751ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
156732026-03-24T21:08:05.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
156742026-03-24T21:08:05.753ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
156752026-03-24T21:08:05.753ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
156762026-03-24T21:08:05.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
156772026-03-24T21:08:05.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
156782026-03-24T21:08:05.757ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
156792026-03-24T21:08:05.757ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
156802026-03-24T21:08:05.757ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
156812026-03-24T21:08:05.757ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
156822026-03-24T21:08:05.757ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
156832026-03-24T21:08:05.757ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 224.304618ms iteration = 67 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
156842026-03-24T21:08:05.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
156852026-03-24T21:08:05.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
156862026-03-24T21:08:05.759ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156872026-03-24T21:08:05.759ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156882026-03-24T21:08:05.759ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156892026-03-24T21:08:05.759ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 212.576342ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
156902026-03-24T21:08:05.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
156912026-03-24T21:08:05.764ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
156922026-03-24T21:08:05.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
156932026-03-24T21:08:05.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
156942026-03-24T21:08:05.765ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
156952026-03-24T21:08:05.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
156962026-03-24T21:08:05.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
156972026-03-24T21:08:05.765ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
156982026-03-24T21:08:05.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
156992026-03-24T21:08:05.766ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
157002026-03-24T21:08:05.766ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
157012026-03-24T21:08:05.767ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
157022026-03-24T21:08:05.767ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
157032026-03-24T21:08:05.767ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
157042026-03-24T21:08:05.767ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
157052026-03-24T21:08:05.767ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
157062026-03-24T21:08:05.767ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
157072026-03-24T21:08:05.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
157082026-03-24T21:08:05.768ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
157092026-03-24T21:08:05.768ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
157102026-03-24T21:08:05.768ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2026-03-24 20:57:07.125641 UTC dns_group = external generation = 2
157112026-03-24T21:08:05.768ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 247.692939ms iteration = 12 status = {"generation":2}
157122026-03-24T21:08:05.768ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
157132026-03-24T21:08:05.768ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
157142026-03-24T21:08:05.768ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 235.760017ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
157152026-03-24T21:08:05.769ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
157162026-03-24T21:08:05.769ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
157172026-03-24T21:08:05.769ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2026-03-24 20:57:05.047625 UTC dns_group = internal generation = 1
157182026-03-24T21:08:05.769ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 253.835575ms iteration = 12 status = {"generation":1}
157192026-03-24T21:08:05.770ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/22eb213d-f2b5-43d3-9c99-dcd13e026df3/state
157202026-03-24T21:08:05.770ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
157212026-03-24T21:08:05.770ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
157222026-03-24T21:08:05.771ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/22eb213d-f2b5-43d3-9c99-dcd13e026df3/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "bdf4b43d-499d-4c64-ae39-c469fa52d284", "content-length": "130", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} })
157232026-03-24T21:08:05.778ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
157242026-03-24T21:08:05.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
157252026-03-24T21:08:05.779ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 175b6065-e38b-47e8-810b-966a3fb83d86 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, generation: Generation(4), time_updated: 2026-03-24T21:07:58.048081635Z }, migration_in: None, migration_out: None } vmm_id = 22eb213d-f2b5-43d3-9c99-dcd13e026df3
157262026-03-24T21:08:05.779ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2754 instance_id = 175b6065-e38b-47e8-810b-966a3fb83d86 migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 22eb213d-f2b5-43d3-9c99-dcd13e026df3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 vmm_id = 22eb213d-f2b5-43d3-9c99-dcd13e026df3 vmm_state = VmmRuntimeState { state: Running, generation: Generation(4), time_updated: 2026-03-24T21:07:58.048081635Z }
157272026-03-24T21:08:05.779ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 175b6065-e38b-47e8-810b-966a3fb83d86 roles = RoleSet { roles: {} } sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 vmm_id = 22eb213d-f2b5-43d3-9c99-dcd13e026df3
157282026-03-24T21:08:05.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher instance_id = 175b6065-e38b-47e8-810b-966a3fb83d86 resource = Database result = Ok(()) sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 vmm_id = 22eb213d-f2b5-43d3-9c99-dcd13e026df3
157292026-03-24T21:08:05.782ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157302026-03-24T21:08:05.785ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
157312026-03-24T21:08:05.785ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
157322026-03-24T21:08:05.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
157332026-03-24T21:08:05.812ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
157342026-03-24T21:08:05.812ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
157352026-03-24T21:08:05.812ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
157362026-03-24T21:08:05.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
157372026-03-24T21:08:05.815ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
157382026-03-24T21:08:05.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
157392026-03-24T21:08:05.817ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
157402026-03-24T21:08:05.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
157412026-03-24T21:08:05.819ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
157422026-03-24T21:08:05.819ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
157432026-03-24T21:08:05.819ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 259.545788ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
157442026-03-24T21:08:05.820ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e95262ce-3e53-4afb-a1ce-d7aa7206e125", "content-length": "680", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} })
157452026-03-24T21:08:05.823ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
157462026-03-24T21:08:05.823ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 263.361691ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
157472026-03-24T21:08:05.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
157482026-03-24T21:08:05.828ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
157492026-03-24T21:08:05.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157502026-03-24T21:08:05.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
157512026-03-24T21:08:05.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157522026-03-24T21:08:05.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157532026-03-24T21:08:05.830ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157542026-03-24T21:08:05.831ZDEBGnexus (BackgroundTasks): listed attached subnets background_task = attached_subnet_manager n_subnets = 0
157552026-03-24T21:08:05.832ZDEBGnexus (BackgroundTasks): activation complete background_task = attached_subnet_manager elapsed = 256.062474ms iteration = 12 status = {"db_error":null,"dendrite":{"switch0":{"errors":[],"n_subnets_added":0,"n_subnets_removed":0,"n_total_subnets":0}},"sled":{}}
157562026-03-24T21:08:05.834ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
157572026-03-24T21:08:05.834ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
157582026-03-24T21:08:05.834ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 301.163071ms iteration = 38 status = {"physical_disks_added":0}
157592026-03-24T21:08:05.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
157602026-03-24T21:08:05.834ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
157612026-03-24T21:08:05.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157622026-03-24T21:08:05.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157632026-03-24T21:08:05.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157642026-03-24T21:08:05.839ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
157652026-03-24T21:08:05.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157662026-03-24T21:08:05.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157672026-03-24T21:08:05.841ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
157682026-03-24T21:08:05.841ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 308.381866ms iteration = 34 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
157692026-03-24T21:08:05.845ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
157702026-03-24T21:08:05.846ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "5f588ce5-eeaa-4de4-b5cf-b82eea845310", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} })
157712026-03-24T21:08:05.846ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
157722026-03-24T21:08:05.846ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
157732026-03-24T21:08:05.846ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 270.196045ms iteration = 12 status = {"errors":[],"probes_by_sled":{"b4293d4f-3308-4dfa-b261-a103215913e4":0}}
157742026-03-24T21:08:05.852ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
157752026-03-24T21:08:05.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
157762026-03-24T21:08:05.859ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:118 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
157772026-03-24T21:08:05.859ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 299.542727ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
157782026-03-24T21:08:05.884ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2026-03-24T20:56:59.957750Z, time_modified: 2026-03-24T20:56:59.957750Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
157792026-03-24T21:08:05.884ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 364.406666ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]}},"default_endpoint":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
157802026-03-24T21:08:05.893ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
157812026-03-24T21:08:05.893ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
157822026-03-24T21:08:05.893ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 333.03478ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
157832026-03-24T21:08:05.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157842026-03-24T21:08:05.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157852026-03-24T21:08:05.897ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = DeployNexusRecords step id = Ensure
157862026-03-24T21:08:05.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
157872026-03-24T21:08:05.897ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
157882026-03-24T21:08:05.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
157892026-03-24T21:08:05.897ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 337.090879ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"running":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
157902026-03-24T21:08:05.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157912026-03-24T21:08:05.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157922026-03-24T21:08:05.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157932026-03-24T21:08:05.901ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
157942026-03-24T21:08:05.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
157952026-03-24T21:08:05.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
157962026-03-24T21:08:05.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157972026-03-24T21:08:05.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
157982026-03-24T21:08:05.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
157992026-03-24T21:08:05.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158002026-03-24T21:08:05.908ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
158012026-03-24T21:08:05.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158022026-03-24T21:08:05.908ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
158032026-03-24T21:08:05.908ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
158042026-03-24T21:08:05.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158052026-03-24T21:08:05.909ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9f903c20-de12-4878-9ef2-4a0b5806f01b", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} })
158062026-03-24T21:08:05.909ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158072026-03-24T21:08:05.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
158082026-03-24T21:08:05.911ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
158092026-03-24T21:08:05.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158102026-03-24T21:08:05.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158112026-03-24T21:08:05.912ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158122026-03-24T21:08:05.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
158132026-03-24T21:08:05.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158142026-03-24T21:08:05.916ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158152026-03-24T21:08:05.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
158162026-03-24T21:08:05.917ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
158172026-03-24T21:08:05.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158182026-03-24T21:08:05.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158192026-03-24T21:08:05.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158202026-03-24T21:08:05.920ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158212026-03-24T21:08:05.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
158222026-03-24T21:08:05.922ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
158232026-03-24T21:08:05.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158242026-03-24T21:08:05.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158252026-03-24T21:08:05.922ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158262026-03-24T21:08:05.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
158272026-03-24T21:08:05.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158282026-03-24T21:08:05.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158292026-03-24T21:08:05.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158302026-03-24T21:08:05.932ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
158312026-03-24T21:08:05.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158322026-03-24T21:08:05.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158332026-03-24T21:08:05.933ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158342026-03-24T21:08:05.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
158352026-03-24T21:08:05.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158362026-03-24T21:08:05.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158372026-03-24T21:08:05.936ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
158382026-03-24T21:08:05.936ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
158392026-03-24T21:08:05.936ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
158402026-03-24T21:08:05.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158412026-03-24T21:08:05.939ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d384ee23-821c-4829-87ed-450169c8299d", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} })
158422026-03-24T21:08:05.939ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 378.803885ms iteration = 34 status = {}
158432026-03-24T21:08:05.939ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 35 reason = Dependency
158442026-03-24T21:08:05.939ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
158452026-03-24T21:08:05.939ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158462026-03-24T21:08:05.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
158472026-03-24T21:08:05.941ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158482026-03-24T21:08:05.942ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
158492026-03-24T21:08:05.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158502026-03-24T21:08:05.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158512026-03-24T21:08:05.945ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
158522026-03-24T21:08:05.945ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158532026-03-24T21:08:05.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
158542026-03-24T21:08:05.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158552026-03-24T21:08:05.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158562026-03-24T21:08:05.947ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158572026-03-24T21:08:05.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158582026-03-24T21:08:05.949ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
158592026-03-24T21:08:05.949ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158602026-03-24T21:08:05.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
158612026-03-24T21:08:05.953ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
158622026-03-24T21:08:05.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158632026-03-24T21:08:05.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158642026-03-24T21:08:05.953ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
158652026-03-24T21:08:05.953ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
158662026-03-24T21:08:05.959ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba252818-9437-4667-91d5-54a9cd315d9e", "content-length": "680", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} })
158672026-03-24T21:08:05.959ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158682026-03-24T21:08:05.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
158692026-03-24T21:08:05.976ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = SledAgent step id = Ensure
158702026-03-24T21:08:05.978ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
158712026-03-24T21:08:05.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158722026-03-24T21:08:05.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158732026-03-24T21:08:05.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158742026-03-24T21:08:05.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158752026-03-24T21:08:05.986ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "76350de8-1a4a-4bfd-b9ca-5ddd159fa299", "date": "Tue, 24 Mar 2026 21:08:05 GMT"} }) sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
158762026-03-24T21:08:05.986ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = FirewallRules step id = Ensure
158772026-03-24T21:08:05.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158782026-03-24T21:08:05.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158792026-03-24T21:08:05.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158802026-03-24T21:08:05.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158812026-03-24T21:08:05.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158822026-03-24T21:08:05.990ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
158832026-03-24T21:08:05.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158842026-03-24T21:08:05.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158852026-03-24T21:08:05.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158862026-03-24T21:08:05.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158872026-03-24T21:08:05.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158882026-03-24T21:08:05.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158892026-03-24T21:08:05.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158902026-03-24T21:08:05.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158912026-03-24T21:08:05.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158922026-03-24T21:08:05.995ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
158932026-03-24T21:08:05.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158942026-03-24T21:08:05.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158952026-03-24T21:08:05.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158962026-03-24T21:08:05.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158972026-03-24T21:08:05.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158982026-03-24T21:08:05.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158992026-03-24T21:08:05.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159002026-03-24T21:08:05.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159012026-03-24T21:08:05.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159022026-03-24T21:08:06.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159032026-03-24T21:08:06.003ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
159042026-03-24T21:08:06.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159052026-03-24T21:08:06.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159062026-03-24T21:08:06.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159072026-03-24T21:08:06.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159082026-03-24T21:08:06.005ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
159092026-03-24T21:08:06.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159102026-03-24T21:08:06.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159112026-03-24T21:08:06.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159122026-03-24T21:08:06.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159132026-03-24T21:08:06.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159142026-03-24T21:08:06.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159152026-03-24T21:08:06.010ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
159162026-03-24T21:08:06.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159172026-03-24T21:08:06.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159182026-03-24T21:08:06.012ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
159192026-03-24T21:08:06.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
159202026-03-24T21:08:06.013ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159212026-03-24T21:08:06.013ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
159222026-03-24T21:08:06.013ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159232026-03-24T21:08:06.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159242026-03-24T21:08:06.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159252026-03-24T21:08:06.013ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159262026-03-24T21:08:06.013ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159272026-03-24T21:08:06.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159282026-03-24T21:08:06.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159292026-03-24T21:08:06.015ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
159302026-03-24T21:08:06.015ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
159312026-03-24T21:08:06.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
159322026-03-24T21:08:06.016ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159332026-03-24T21:08:06.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159342026-03-24T21:08:06.017ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159352026-03-24T21:08:06.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159362026-03-24T21:08:06.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159372026-03-24T21:08:06.019ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
159382026-03-24T21:08:06.019ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
159392026-03-24T21:08:06.019ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
159402026-03-24T21:08:06.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
159412026-03-24T21:08:06.019ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159422026-03-24T21:08:06.019ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159432026-03-24T21:08:06.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159442026-03-24T21:08:06.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159452026-03-24T21:08:06.020ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159462026-03-24T21:08:06.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159472026-03-24T21:08:06.021ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159482026-03-24T21:08:06.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159492026-03-24T21:08:06.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159502026-03-24T21:08:06.022ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
159512026-03-24T21:08:06.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159522026-03-24T21:08:06.023ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "689f44b6-cf6c-4616-81ec-5446f2731766", "content-length": "938", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
159532026-03-24T21:08:06.023ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: fddb:9ff3:dfd5::5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(13618710) }, VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(13618710) }} sled = buskin
159542026-03-24T21:08:06.023ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
159552026-03-24T21:08:06.023ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
159562026-03-24T21:08:06.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159572026-03-24T21:08:06.023ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 490.65031ms iteration = 23 status = {}
159582026-03-24T21:08:06.025ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
159592026-03-24T21:08:06.025ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159602026-03-24T21:08:06.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159612026-03-24T21:08:06.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159622026-03-24T21:08:06.027ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
159632026-03-24T21:08:06.027ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
159642026-03-24T21:08:06.027ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1350be45-5bd1-4f9f-9dad-cc0ed4169616", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
159652026-03-24T21:08:06.027ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159662026-03-24T21:08:06.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159672026-03-24T21:08:06.031ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159682026-03-24T21:08:06.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159692026-03-24T21:08:06.033ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
159702026-03-24T21:08:06.033ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159712026-03-24T21:08:06.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159722026-03-24T21:08:06.035ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159732026-03-24T21:08:06.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159742026-03-24T21:08:06.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159752026-03-24T21:08:06.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159762026-03-24T21:08:06.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159772026-03-24T21:08:06.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159782026-03-24T21:08:06.039ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159792026-03-24T21:08:06.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159802026-03-24T21:08:06.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159812026-03-24T21:08:06.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159822026-03-24T21:08:06.041ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159832026-03-24T21:08:06.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159842026-03-24T21:08:06.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159852026-03-24T21:08:06.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159862026-03-24T21:08:06.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159872026-03-24T21:08:06.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159882026-03-24T21:08:06.045ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159892026-03-24T21:08:06.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159902026-03-24T21:08:06.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159912026-03-24T21:08:06.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159922026-03-24T21:08:06.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159932026-03-24T21:08:06.049ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
159942026-03-24T21:08:06.049ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
159952026-03-24T21:08:06.049ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
159962026-03-24T21:08:06.049ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5cd96e79-189e-498c-9263-abdf92659d08", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
159972026-03-24T21:08:06.049ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 110.593752ms iteration = 35 status = {}
159982026-03-24T21:08:06.050ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9ddb3bbd-073c-4fa8-be5b-aee390876e3a", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
159992026-03-24T21:08:06.050ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
160002026-03-24T21:08:06.050ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
160012026-03-24T21:08:06.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
160022026-03-24T21:08:06.053ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
160032026-03-24T21:08:06.053ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160042026-03-24T21:08:06.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160052026-03-24T21:08:06.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160062026-03-24T21:08:06.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160072026-03-24T21:08:06.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160082026-03-24T21:08:06.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160092026-03-24T21:08:06.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160102026-03-24T21:08:06.063ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
160112026-03-24T21:08:06.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160122026-03-24T21:08:06.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160132026-03-24T21:08:06.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160142026-03-24T21:08:06.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160152026-03-24T21:08:06.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160162026-03-24T21:08:06.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160172026-03-24T21:08:06.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160182026-03-24T21:08:06.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160192026-03-24T21:08:06.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160202026-03-24T21:08:06.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160212026-03-24T21:08:06.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160222026-03-24T21:08:06.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160232026-03-24T21:08:06.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160242026-03-24T21:08:06.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160252026-03-24T21:08:06.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160262026-03-24T21:08:06.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160272026-03-24T21:08:06.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160282026-03-24T21:08:06.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
160292026-03-24T21:08:06.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160302026-03-24T21:08:06.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160312026-03-24T21:08:06.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160322026-03-24T21:08:06.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160332026-03-24T21:08:06.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160342026-03-24T21:08:06.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160352026-03-24T21:08:06.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160362026-03-24T21:08:06.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160372026-03-24T21:08:06.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160382026-03-24T21:08:06.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160392026-03-24T21:08:06.084ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
160402026-03-24T21:08:06.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160412026-03-24T21:08:06.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160422026-03-24T21:08:06.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160432026-03-24T21:08:06.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160442026-03-24T21:08:06.087ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
160452026-03-24T21:08:06.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160462026-03-24T21:08:06.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160472026-03-24T21:08:06.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160482026-03-24T21:08:06.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160492026-03-24T21:08:06.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160502026-03-24T21:08:06.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160512026-03-24T21:08:06.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160522026-03-24T21:08:06.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
160532026-03-24T21:08:06.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160542026-03-24T21:08:06.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160552026-03-24T21:08:06.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160562026-03-24T21:08:06.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160572026-03-24T21:08:06.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160582026-03-24T21:08:06.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160592026-03-24T21:08:06.097ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
160602026-03-24T21:08:06.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160612026-03-24T21:08:06.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
160622026-03-24T21:08:06.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160632026-03-24T21:08:06.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160642026-03-24T21:08:06.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160652026-03-24T21:08:06.098ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
160662026-03-24T21:08:06.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
160672026-03-24T21:08:06.101ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
160682026-03-24T21:08:06.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
160692026-03-24T21:08:06.103ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
160702026-03-24T21:08:06.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160712026-03-24T21:08:06.103ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
160722026-03-24T21:08:06.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160732026-03-24T21:08:06.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
160742026-03-24T21:08:06.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160752026-03-24T21:08:06.105ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 572.794026ms iteration = 23 status = {}
160762026-03-24T21:08:06.108ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
160772026-03-24T21:08:06.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160782026-03-24T21:08:06.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160792026-03-24T21:08:06.110ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
160802026-03-24T21:08:06.110ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
160812026-03-24T21:08:06.110ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
160822026-03-24T21:08:06.146ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
160832026-03-24T21:08:06.146ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
160842026-03-24T21:08:06.146ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 34944d2f-5f99-4929-ad6f-b606df3c1bd9 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
160852026-03-24T21:08:06.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160862026-03-24T21:08:06.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160872026-03-24T21:08:06.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160882026-03-24T21:08:06.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
160892026-03-24T21:08:06.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160902026-03-24T21:08:06.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160912026-03-24T21:08:06.154ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
160922026-03-24T21:08:06.154ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
160932026-03-24T21:08:06.154ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 621.377096ms iteration = 29 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
160942026-03-24T21:08:06.276ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
160952026-03-24T21:08:06.276ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
160962026-03-24T21:08:06.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160972026-03-24T21:08:06.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160982026-03-24T21:08:06.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160992026-03-24T21:08:06.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161002026-03-24T21:08:06.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161012026-03-24T21:08:06.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
161022026-03-24T21:08:06.285ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
161032026-03-24T21:08:06.285ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "4687f63d-5b7f-4e80-92be-6bf4874a5615", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
161042026-03-24T21:08:06.285ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
161052026-03-24T21:08:06.285ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = Dns step id = Ensure
161062026-03-24T21:08:06.285ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
161072026-03-24T21:08:06.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161082026-03-24T21:08:06.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161092026-03-24T21:08:06.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161102026-03-24T21:08:06.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161112026-03-24T21:08:06.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161122026-03-24T21:08:06.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161132026-03-24T21:08:06.292ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
161142026-03-24T21:08:06.292ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
161152026-03-24T21:08:06.292ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
161162026-03-24T21:08:06.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161172026-03-24T21:08:06.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161182026-03-24T21:08:06.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161192026-03-24T21:08:06.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161202026-03-24T21:08:06.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161212026-03-24T21:08:06.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161222026-03-24T21:08:06.299ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
161232026-03-24T21:08:06.299ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
161242026-03-24T21:08:06.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161252026-03-24T21:08:06.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161262026-03-24T21:08:06.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161272026-03-24T21:08:06.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161282026-03-24T21:08:06.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161292026-03-24T21:08:06.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161302026-03-24T21:08:06.307ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
161312026-03-24T21:08:06.307ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
161322026-03-24T21:08:06.307ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
161332026-03-24T21:08:06.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161342026-03-24T21:08:06.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161352026-03-24T21:08:06.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161362026-03-24T21:08:06.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161372026-03-24T21:08:06.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161382026-03-24T21:08:06.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161392026-03-24T21:08:06.342ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 4
161402026-03-24T21:08:06.342ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221
161412026-03-24T21:08:06.344ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
161422026-03-24T21:08:06.344ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:53937
161432026-03-24T21:08:06.345ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:53937 req_id = e601b829-89b2-4b39-8f2f-50e3da403a90 uri = /metrics/producers version = 1.0.0
161442026-03-24T21:08:06.345ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:53937 req_id = e601b829-89b2-4b39-8f2f-50e3da403a90 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
161452026-03-24T21:08:06.345ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:53937 req_id = e601b829-89b2-4b39-8f2f-50e3da403a90 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
161462026-03-24T21:08:06.361ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
161472026-03-24T21:08:06.361ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
161482026-03-24T21:08:06.361ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
161492026-03-24T21:08:06.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161502026-03-24T21:08:06.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161512026-03-24T21:08:06.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161522026-03-24T21:08:06.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161532026-03-24T21:08:06.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161542026-03-24T21:08:06.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161552026-03-24T21:08:06.369ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
161562026-03-24T21:08:06.369ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
161572026-03-24T21:08:06.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161582026-03-24T21:08:06.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161592026-03-24T21:08:06.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161602026-03-24T21:08:06.376ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161612026-03-24T21:08:06.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161622026-03-24T21:08:06.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161632026-03-24T21:08:06.378ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
161642026-03-24T21:08:06.378ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
161652026-03-24T21:08:06.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161662026-03-24T21:08:06.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161672026-03-24T21:08:06.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161682026-03-24T21:08:06.382ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
161692026-03-24T21:08:06.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161702026-03-24T21:08:06.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161712026-03-24T21:08:06.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161722026-03-24T21:08:06.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161732026-03-24T21:08:06.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161742026-03-24T21:08:06.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161752026-03-24T21:08:06.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161762026-03-24T21:08:06.390ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161772026-03-24T21:08:06.392ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
161782026-03-24T21:08:06.392ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
161792026-03-24T21:08:06.392ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = OmicronZones step id = Cleanup
161802026-03-24T21:08:06.392ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = OmicronZones step id = Cleanup
161812026-03-24T21:08:06.392ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = PhysicalDisks step id = Cleanup
161822026-03-24T21:08:06.392ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = Clickhouse step id = Ensure
161832026-03-24T21:08:06.392ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = Clickhouse step id = Ensure
161842026-03-24T21:08:06.393ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
161852026-03-24T21:08:06.398ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "2a223236-1d6e-424e-9675-33402c3aedc3", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
161862026-03-24T21:08:06.398ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = SupportBundles step id = Cleanup
161872026-03-24T21:08:06.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161882026-03-24T21:08:06.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161892026-03-24T21:08:06.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161902026-03-24T21:08:06.402ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
161912026-03-24T21:08:06.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161922026-03-24T21:08:06.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161932026-03-24T21:08:06.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161942026-03-24T21:08:06.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161952026-03-24T21:08:06.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161962026-03-24T21:08:06.408ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
161972026-03-24T21:08:06.428ZINFOnexus: assigned collector to new producer collector_id = a309df7c-2a20-4201-8e80-eb9f7e355177 file = nexus/src/app/oximeter.rs:76 producer_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb
161982026-03-24T21:08:06.428ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 82962 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:53937 req_id = e601b829-89b2-4b39-8f2f-50e3da403a90 response_code = 201 uri = /metrics/producers
161992026-03-24T21:08:06.428ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "e601b829-89b2-4b39-8f2f-50e3da403a90", "content-length": "41", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162002026-03-24T21:08:06.428ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
162012026-03-24T21:08:06.454ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = OmicronZones step id = Cleanup
162022026-03-24T21:08:06.454ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
162032026-03-24T21:08:06.454ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
162042026-03-24T21:08:06.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162052026-03-24T21:08:06.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162062026-03-24T21:08:06.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162072026-03-24T21:08:06.458ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
162082026-03-24T21:08:06.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162092026-03-24T21:08:06.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162102026-03-24T21:08:06.498ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
162112026-03-24T21:08:06.498ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9e1c09c1-3960-4ecf-a904-19e137699ab3 (blueprint), false)
162122026-03-24T21:08:06.498ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = Cockroach step id = Ensure
162132026-03-24T21:08:06.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162142026-03-24T21:08:06.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162152026-03-24T21:08:06.533ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
162162026-03-24T21:08:06.533ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a263453d-b162-4a4e-8e3d-17d316564725 step component = MgsUpdates step id = Ensure
162172026-03-24T21:08:06.533ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 35 reason = Dependency
162182026-03-24T21:08:06.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 15 reason = Dependency
162192026-03-24T21:08:06.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
162202026-03-24T21:08:06.533ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
162212026-03-24T21:08:06.534ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.012637196s iteration = 13 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":2760,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":171278039,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":171277999,"secs":0}},"event_index":1,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":171296539,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":204205227,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":204205177,"secs":0}},"event_index":2,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":375597318,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":79860373,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":79860323,"secs":0}},"event_index":3,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":455469831,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":9250279,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":9250239,"secs":0}},"event_index":4,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":464739611,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":299748472,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":299748442,"secs":0}},"event_index":5,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":764501084,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":106333236,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":106333186,"secs":0}},"event_index":6,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":870841721,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6361,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":6331,"secs":0}},"event_index":7,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":870860782,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2640,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2610,"secs":0}},"event_index":8,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":870866072,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3510,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3470,"secs":0}},"event_index":9,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":870871042,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1740,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1710,"secs":0}},"event_index":10,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":870874332,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6391463,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":6391423,"secs":0}},"event_index":11,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":877267375,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":55906707,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":55906677,"secs":0}},"event_index":12,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":933183082,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":44246373,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":44246343,"secs":0}},"event_index":13,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":977448256,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":34862970,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":34862930,"secs":0}},"event_index":14,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":12325156,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5970,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5930,"secs":0}},"event_index":15,"execution_id":"24bfca19-e923-4c75-8dc4-ce2bc0ae409d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":12338617,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3"}
162222026-03-24T21:08:06.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
162232026-03-24T21:08:06.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
162242026-03-24T21:08:06.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162252026-03-24T21:08:06.535ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162262026-03-24T21:08:06.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
162272026-03-24T21:08:06.538ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
162282026-03-24T21:08:06.538ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
162292026-03-24T21:08:06.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
162302026-03-24T21:08:06.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
162312026-03-24T21:08:06.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
162322026-03-24T21:08:06.540ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
162332026-03-24T21:08:06.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
162342026-03-24T21:08:06.542ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162352026-03-24T21:08:06.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
162362026-03-24T21:08:06.543ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
162372026-03-24T21:08:06.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
162382026-03-24T21:08:06.573ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(790f3085-8870-4f41-8538-c1fd8911c833, 0), (da877b83-376f-4ce2-a60a-42f1dd57b87c, 0), (da898c00-a280-40a9-aa3c-5dfd1b54176d, 0)]
162392026-03-24T21:08:06.573ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1472, tv_nsec: 787794898 } })
162402026-03-24T21:08:06.575ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1345db7a-abc1-4a4c-a49c-e6c56102e521.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1920db37-87f5-4a96-b3ef-1db2eb59e9a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6b02d2eb-4373-4f12-8efc-4c0e3f4fa77b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("709c4704-f768-4640-a198-a23621ed6e11.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1472, tv_nsec: 789572246 } })
162412026-03-24T21:08:06.576ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
162422026-03-24T21:08:06.576ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
162432026-03-24T21:08:06.576ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
162442026-03-24T21:08:06.577ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "9db1f0cc-e26a-4294-b416-1768e234b8a7", "content-length": "476", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162452026-03-24T21:08:06.577ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
162462026-03-24T21:08:06.577ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d63008a-042e-4952-bb70-1ef447c123d6", "content-length": "734", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162472026-03-24T21:08:06.577ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
162482026-03-24T21:08:06.577ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "00aa1522-7cf7-46af-b7b1-fc86b5ba9a86", "content-length": "111", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162492026-03-24T21:08:06.577ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
162502026-03-24T21:08:06.578ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6292b847-3102-4cb6-b3cf-a724d0b24128", "content-length": "111", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162512026-03-24T21:08:06.578ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
162522026-03-24T21:08:06.579ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a80f38e7-b768-4ba3-a5fb-693576fca701", "content-length": "163", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162532026-03-24T21:08:06.579ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
162542026-03-24T21:08:06.579ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a2294e2-245f-4fa3-9a50-8045f84a8e9d", "content-length": "163", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162552026-03-24T21:08:06.579ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
162562026-03-24T21:08:06.580ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c98273c1-16a7-428e-9b25-b749c277729f", "content-length": "166", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162572026-03-24T21:08:06.580ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
162582026-03-24T21:08:06.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "52b3c0b0-8169-492f-b466-7a683d1c1398", "content-length": "166", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162592026-03-24T21:08:06.581ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
162602026-03-24T21:08:06.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4dae5138-e3d4-43dc-b83c-9f7a3fc89cb2", "content-length": "702", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162612026-03-24T21:08:06.581ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
162622026-03-24T21:08:06.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7dda2e59-0355-4089-888b-afa994887c76", "content-length": "727", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162632026-03-24T21:08:06.581ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
162642026-03-24T21:08:06.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7d4948d-2c61-46d6-a315-e2c051662697", "content-length": "729", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162652026-03-24T21:08:06.581ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
162662026-03-24T21:08:06.582ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a85638b5-3fd2-4a4d-a6e2-72184563139b", "content-length": "728", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162672026-03-24T21:08:06.582ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
162682026-03-24T21:08:06.582ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "01e8ffb9-c855-42ba-90a7-1f8ccc1525cb", "content-length": "722", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162692026-03-24T21:08:06.582ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
162702026-03-24T21:08:06.582ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "16d027a5-5116-4a49-846b-007721f4049f", "content-length": "10", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162712026-03-24T21:08:06.582ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
162722026-03-24T21:08:06.582ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "de136bf6-e90a-4a03-9245-a1f9b00451fc", "content-length": "144", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162732026-03-24T21:08:06.582ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
162742026-03-24T21:08:06.583ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2451b7a3-e71e-4d76-8a33-1e04fa8ac28e", "content-length": "147", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162752026-03-24T21:08:06.583ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
162762026-03-24T21:08:06.583ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d2b48f3-d545-40da-b928-88ff5859c02e", "content-length": "109", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162772026-03-24T21:08:06.583ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
162782026-03-24T21:08:06.584ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "871365b2-5d08-42fd-bb0b-95411924c82e", "content-length": "109", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162792026-03-24T21:08:06.584ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
162802026-03-24T21:08:06.584ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "be85a32f-39a2-4606-a51c-a51b6bcca0b4", "content-length": "163", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162812026-03-24T21:08:06.584ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
162822026-03-24T21:08:06.585ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fbd12ff-92f0-4334-b83b-186e0689e80e", "content-length": "163", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162832026-03-24T21:08:06.585ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
162842026-03-24T21:08:06.585ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f50f573-5a88-42b2-ab2b-65d5f3388eaf", "content-length": "166", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162852026-03-24T21:08:06.585ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
162862026-03-24T21:08:06.586ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe02d497-b3aa-4694-8345-b34b9739ceea", "content-length": "166", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162872026-03-24T21:08:06.586ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
162882026-03-24T21:08:06.586ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb6d1ae7-768d-4514-b5c5-11ba876e27f1", "content-length": "702", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162892026-03-24T21:08:06.586ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
162902026-03-24T21:08:06.586ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "acaa8c5d-8181-4dc8-9794-561ae8a5dcd4", "content-length": "727", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162912026-03-24T21:08:06.586ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
162922026-03-24T21:08:06.587ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c138517-9064-4b0b-b044-ecea422d9da0", "content-length": "729", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162932026-03-24T21:08:06.587ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
162942026-03-24T21:08:06.587ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a25ea24-c085-498c-bb30-1f2b40d1eed6", "content-length": "728", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
162952026-03-24T21:08:06.587ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
162962026-03-24T21:08:06.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
162972026-03-24T21:08:06.589ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162982026-03-24T21:08:06.591ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
162992026-03-24T21:08:06.591ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163002026-03-24T21:08:06.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
163012026-03-24T21:08:06.593ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163022026-03-24T21:08:06.593ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
163032026-03-24T21:08:06.593ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 59.250548ms iteration = 35 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
163042026-03-24T21:08:06.594ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
163052026-03-24T21:08:06.594ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163062026-03-24T21:08:06.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
163072026-03-24T21:08:06.694ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
163082026-03-24T21:08:06.694ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
163092026-03-24T21:08:06.779ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "8107be86-acce-4064-b143-b93ebb5b642f", "content-length": "110909", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163102026-03-24T21:08:06.780ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
163112026-03-24T21:08:06.780ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
163122026-03-24T21:08:06.780ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
163132026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
163142026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
163152026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
163162026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
163172026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
163182026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "469c4deb-626d-4a21-b46e-079b296b9688", "content-length": "64", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163192026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "52766324-13f4-481a-afd3-90a2394ec464", "content-length": "64", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163202026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "38fa50d1-ec95-492c-be27-ef33cc669c94", "content-length": "64", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163212026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3b8fd5b-9edd-4647-abda-0e06da92456f", "content-length": "64", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163222026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "584501ba-beb9-4420-8b90-6fc4ec4bc2d5", "content-length": "64", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163232026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
163242026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
163252026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
163262026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
163272026-03-24T21:08:06.780ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
163282026-03-24T21:08:06.787ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "54695586-c0c8-4430-a4db-6e35755b63a0", "content-length": "409853", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163292026-03-24T21:08:06.788ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "48063f7b-9f74-40ab-a4f7-5a3abcd313c0", "content-length": "405618", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163302026-03-24T21:08:06.790ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e664b7b3-c85c-416f-ba66-2121de547aff", "content-length": "409885", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163312026-03-24T21:08:06.793ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e968831d-b0c8-4fb7-85c3-5e37d448b19b", "content-length": "436599", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163322026-03-24T21:08:06.793ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "57c7027a-0819-454b-88b2-9093f470fe40", "content-length": "417608", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163332026-03-24T21:08:06.796ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
163342026-03-24T21:08:06.796ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
163352026-03-24T21:08:06.796ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
163362026-03-24T21:08:06.796ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
163372026-03-24T21:08:06.796ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
163382026-03-24T21:08:06.797ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 (omicron_zone)
163392026-03-24T21:08:06.797ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
163402026-03-24T21:08:06.800ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "07673624-1457-424c-a7aa-55c2993ece64", "content-length": "127", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163412026-03-24T21:08:06.800ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
163422026-03-24T21:08:06.803ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
163432026-03-24T21:08:06.803ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
163442026-03-24T21:08:06.803ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "763fb55b-809e-495e-8ebc-24651b0693f6", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163452026-03-24T21:08:06.803ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
163462026-03-24T21:08:06.803ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
163472026-03-24T21:08:06.803ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
163482026-03-24T21:08:06.804ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "106343bb-a051-4b4e-b861-3256ae1e2c30", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163492026-03-24T21:08:06.804ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
163502026-03-24T21:08:06.804ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
163512026-03-24T21:08:06.804ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
163522026-03-24T21:08:06.804ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b38d1184-be0f-422e-9e52-2dd22272ec3c", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:08:06 GMT"} })
163532026-03-24T21:08:06.805ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
163542026-03-24T21:08:06.805ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
163552026-03-24T21:08:06.805ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
163562026-03-24T21:08:06.805ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163572026-03-24T21:08:06.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
163582026-03-24T21:08:06.807ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163592026-03-24T21:08:06.809ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
163602026-03-24T21:08:06.809ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163612026-03-24T21:08:06.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
163622026-03-24T21:08:07.055ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 9b8824f4-eaf4-42c4-80a6-3c8210ba299f file = nexus/db-queries/src/db/datastore/inventory.rs:1757
163632026-03-24T21:08:07.055ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 9b8824f4-eaf4-42c4-80a6-3c8210ba299f time_started = 2026-03-24 21:08:06.576442 UTC
163642026-03-24T21:08:07.055ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 60 reason = Dependency
163652026-03-24T21:08:07.055ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
163662026-03-24T21:08:07.055ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 521.646526ms iteration = 15 status = {"collection_id":"9b8824f4-eaf4-42c4-80a6-3c8210ba299f","time_done":"2026-03-24 21:08:06.805125 UTC","time_started":"2026-03-24 21:08:06.576442 UTC"}
163672026-03-24T21:08:07.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
163682026-03-24T21:08:07.057ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163692026-03-24T21:08:07.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
163702026-03-24T21:08:07.060ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
163712026-03-24T21:08:07.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
163722026-03-24T21:08:07.062ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
163732026-03-24T21:08:07.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
163742026-03-24T21:08:07.105ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 50.131152ms iteration = 60 status = {"Loaded":{"collection_id":"fae51d14-063d-4c09-aff4-5b856f69c82a","time_loaded":"2026-03-24T21:08:07.055429641Z","time_started":"2026-03-24T21:08:06.582692Z"}}
163752026-03-24T21:08:07.105ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 19 reason = Dependency
163762026-03-24T21:08:07.105ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 28 reason = Dependency
163772026-03-24T21:08:07.105ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
163782026-03-24T21:08:07.105ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 30 reason = Dependency
163792026-03-24T21:08:07.105ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 7.801\u{b5}s iteration = 28 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
163802026-03-24T21:08:07.105ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 39 reason = Dependency
163812026-03-24T21:08:07.105ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
163822026-03-24T21:08:07.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
163832026-03-24T21:08:07.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
163842026-03-24T21:08:07.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
163852026-03-24T21:08:07.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
163862026-03-24T21:08:07.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
163872026-03-24T21:08:07.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163882026-03-24T21:08:07.107ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163892026-03-24T21:08:07.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163902026-03-24T21:08:07.110ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
163912026-03-24T21:08:07.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
163922026-03-24T21:08:07.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
163932026-03-24T21:08:07.110ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
163942026-03-24T21:08:07.110ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
163952026-03-24T21:08:07.110ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
163962026-03-24T21:08:07.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
163972026-03-24T21:08:07.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
163982026-03-24T21:08:07.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
163992026-03-24T21:08:07.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164002026-03-24T21:08:07.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164012026-03-24T21:08:07.112ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
164022026-03-24T21:08:07.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
164032026-03-24T21:08:07.114ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164042026-03-24T21:08:07.114ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
164052026-03-24T21:08:07.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164062026-03-24T21:08:07.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164072026-03-24T21:08:07.115ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164082026-03-24T21:08:07.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164092026-03-24T21:08:07.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164102026-03-24T21:08:07.116ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
164112026-03-24T21:08:07.116ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
164122026-03-24T21:08:07.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
164132026-03-24T21:08:07.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164142026-03-24T21:08:07.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164152026-03-24T21:08:07.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164162026-03-24T21:08:07.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164172026-03-24T21:08:07.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164182026-03-24T21:08:07.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164192026-03-24T21:08:07.121ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
164202026-03-24T21:08:07.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164212026-03-24T21:08:07.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164222026-03-24T21:08:07.121ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164232026-03-24T21:08:07.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164242026-03-24T21:08:07.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164252026-03-24T21:08:07.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164262026-03-24T21:08:07.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164272026-03-24T21:08:07.125ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
164282026-03-24T21:08:07.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164292026-03-24T21:08:07.126ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164302026-03-24T21:08:07.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164312026-03-24T21:08:07.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164322026-03-24T21:08:07.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164332026-03-24T21:08:07.128ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
164342026-03-24T21:08:07.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164352026-03-24T21:08:07.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164362026-03-24T21:08:07.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164372026-03-24T21:08:07.132ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164382026-03-24T21:08:07.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164392026-03-24T21:08:07.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164402026-03-24T21:08:07.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164412026-03-24T21:08:07.139ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164422026-03-24T21:08:07.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164432026-03-24T21:08:07.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164442026-03-24T21:08:07.142ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
164452026-03-24T21:08:07.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164462026-03-24T21:08:07.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164472026-03-24T21:08:07.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164482026-03-24T21:08:07.147ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164492026-03-24T21:08:07.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164502026-03-24T21:08:07.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164512026-03-24T21:08:07.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164522026-03-24T21:08:07.151ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164532026-03-24T21:08:07.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164542026-03-24T21:08:07.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164552026-03-24T21:08:07.153ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
164562026-03-24T21:08:07.153ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 48.162819ms iteration = 19 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"fae51d14-063d-4c09-aff4-5b856f69c82a","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
164572026-03-24T21:08:07.179ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 73.42688ms iteration = 39 status = {"physical_disks_added":0}
164582026-03-24T21:08:07.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164592026-03-24T21:08:07.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164602026-03-24T21:08:07.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164612026-03-24T21:08:07.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164622026-03-24T21:08:07.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164632026-03-24T21:08:07.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
164642026-03-24T21:08:07.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164652026-03-24T21:08:07.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164662026-03-24T21:08:07.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164672026-03-24T21:08:07.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164682026-03-24T21:08:07.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164692026-03-24T21:08:07.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
164702026-03-24T21:08:07.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164712026-03-24T21:08:07.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164722026-03-24T21:08:07.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164732026-03-24T21:08:07.200ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
164742026-03-24T21:08:07.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164752026-03-24T21:08:07.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164762026-03-24T21:08:07.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164772026-03-24T21:08:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164782026-03-24T21:08:07.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164792026-03-24T21:08:07.207ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
164802026-03-24T21:08:07.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164812026-03-24T21:08:07.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164822026-03-24T21:08:07.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164832026-03-24T21:08:07.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164842026-03-24T21:08:07.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164852026-03-24T21:08:07.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
164862026-03-24T21:08:07.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164872026-03-24T21:08:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164882026-03-24T21:08:07.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164892026-03-24T21:08:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164902026-03-24T21:08:07.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164912026-03-24T21:08:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
164922026-03-24T21:08:07.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164932026-03-24T21:08:07.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164942026-03-24T21:08:07.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164952026-03-24T21:08:07.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164962026-03-24T21:08:07.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164972026-03-24T21:08:07.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164982026-03-24T21:08:07.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164992026-03-24T21:08:07.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
165002026-03-24T21:08:07.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165012026-03-24T21:08:07.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165022026-03-24T21:08:07.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165032026-03-24T21:08:07.247ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
165042026-03-24T21:08:07.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165052026-03-24T21:08:07.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165062026-03-24T21:08:07.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165072026-03-24T21:08:07.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165082026-03-24T21:08:07.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165092026-03-24T21:08:07.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165102026-03-24T21:08:07.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165112026-03-24T21:08:07.256ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
165122026-03-24T21:08:07.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165132026-03-24T21:08:07.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165142026-03-24T21:08:07.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165152026-03-24T21:08:07.261ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
165162026-03-24T21:08:07.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165172026-03-24T21:08:07.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165182026-03-24T21:08:07.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165192026-03-24T21:08:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165202026-03-24T21:08:07.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165212026-03-24T21:08:07.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165222026-03-24T21:08:07.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165232026-03-24T21:08:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
165242026-03-24T21:08:07.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165252026-03-24T21:08:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165262026-03-24T21:08:07.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165272026-03-24T21:08:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165282026-03-24T21:08:07.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165292026-03-24T21:08:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
165302026-03-24T21:08:07.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165312026-03-24T21:08:07.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165322026-03-24T21:08:07.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165332026-03-24T21:08:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165342026-03-24T21:08:07.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165352026-03-24T21:08:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
165362026-03-24T21:08:07.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165372026-03-24T21:08:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165382026-03-24T21:08:07.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165392026-03-24T21:08:07.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165402026-03-24T21:08:07.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165412026-03-24T21:08:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
165422026-03-24T21:08:07.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165432026-03-24T21:08:07.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165442026-03-24T21:08:07.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165452026-03-24T21:08:07.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165462026-03-24T21:08:07.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165472026-03-24T21:08:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
165482026-03-24T21:08:07.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165492026-03-24T21:08:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165502026-03-24T21:08:07.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165512026-03-24T21:08:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165522026-03-24T21:08:07.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165532026-03-24T21:08:07.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
165542026-03-24T21:08:07.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165552026-03-24T21:08:07.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165562026-03-24T21:08:07.349ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
165572026-03-24T21:08:07.349ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
165582026-03-24T21:08:07.349ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 03eae9f2-0e75-445c-8645-2aaf910916b6 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
165592026-03-24T21:08:07.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165602026-03-24T21:08:07.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165612026-03-24T21:08:07.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165622026-03-24T21:08:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
165632026-03-24T21:08:07.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165642026-03-24T21:08:07.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165652026-03-24T21:08:07.356ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
165662026-03-24T21:08:07.356ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
165672026-03-24T21:08:07.356ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 250.968008ms iteration = 30 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
165682026-03-24T21:08:08.560ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2895 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 253105d5-18eb-4942-aa8b-080992acd837 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
165692026-03-24T21:08:10.532ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 134 reason = Timeout
165702026-03-24T21:08:10.532ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
165712026-03-24T21:08:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
165722026-03-24T21:08:10.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165732026-03-24T21:08:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
165742026-03-24T21:08:10.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
165752026-03-24T21:08:10.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
165762026-03-24T21:08:10.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
165772026-03-24T21:08:10.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.300767ms iteration = 134 status = {"config_updated":false}
165782026-03-24T21:08:15.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
165792026-03-24T21:08:15.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 135 reason = Timeout
165802026-03-24T21:08:15.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
165812026-03-24T21:08:15.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
165822026-03-24T21:08:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
165832026-03-24T21:08:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
165842026-03-24T21:08:15.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165852026-03-24T21:08:15.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165862026-03-24T21:08:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
165872026-03-24T21:08:15.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
165882026-03-24T21:08:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
165892026-03-24T21:08:15.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
165902026-03-24T21:08:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
165912026-03-24T21:08:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
165922026-03-24T21:08:15.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
165932026-03-24T21:08:15.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
165942026-03-24T21:08:15.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
165952026-03-24T21:08:15.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.360755ms iteration = 135 status = {"config_updated":false}
165962026-03-24T21:08:15.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165972026-03-24T21:08:15.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
165982026-03-24T21:08:15.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
165992026-03-24T21:08:15.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
166002026-03-24T21:08:15.558ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
166012026-03-24T21:08:15.558ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.346134ms iteration = 68 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
166022026-03-24T21:08:15.560ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
166032026-03-24T21:08:15.560ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
166042026-03-24T21:08:15.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
166052026-03-24T21:08:15.562ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
166062026-03-24T21:08:15.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
166072026-03-24T21:08:15.563ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.026612ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
166082026-03-24T21:08:18.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 577 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 7015ad16-f417-4033-b2fa-bfcb5598863d response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
166092026-03-24T21:08:20.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 61 reason = Timeout
166102026-03-24T21:08:20.534ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 136 reason = Timeout
166112026-03-24T21:08:20.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
166122026-03-24T21:08:20.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166132026-03-24T21:08:20.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
166142026-03-24T21:08:20.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166152026-03-24T21:08:20.536ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166162026-03-24T21:08:20.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166172026-03-24T21:08:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
166182026-03-24T21:08:20.539ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
166192026-03-24T21:08:20.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
166202026-03-24T21:08:20.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
166212026-03-24T21:08:20.540ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166222026-03-24T21:08:20.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166232026-03-24T21:08:20.541ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = fae51d14-063d-4c09-aff4-5b856f69c82a original_time_started = 2026-03-24 21:08:06.582692 UTC
166242026-03-24T21:08:20.541ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.736089ms iteration = 61 status = {"Loaded":{"collection_id":"fae51d14-063d-4c09-aff4-5b856f69c82a","time_loaded":"2026-03-24T21:08:20.533949856Z","time_started":"2026-03-24T21:08:06.582692Z"}}
166252026-03-24T21:08:20.542ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
166262026-03-24T21:08:20.542ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.125809ms iteration = 136 status = {"config_updated":false}
166272026-03-24T21:08:20.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 46 reason = Timeout
166282026-03-24T21:08:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
166292026-03-24T21:08:20.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 46 reason = Timeout
166302026-03-24T21:08:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
166312026-03-24T21:08:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
166322026-03-24T21:08:20.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
166332026-03-24T21:08:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
166342026-03-24T21:08:20.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
166352026-03-24T21:08:20.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166362026-03-24T21:08:20.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166372026-03-24T21:08:20.585ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
166382026-03-24T21:08:20.585ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
166392026-03-24T21:08:20.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
166402026-03-24T21:08:20.586ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
166412026-03-24T21:08:20.587ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.319514ms iteration = 46 status = "NoSitrep"
166422026-03-24T21:08:20.588ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
166432026-03-24T21:08:20.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
166442026-03-24T21:08:20.588ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
166452026-03-24T21:08:20.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
166462026-03-24T21:08:20.590ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166472026-03-24T21:08:20.591ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
166482026-03-24T21:08:20.594ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
166492026-03-24T21:08:20.594ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
166502026-03-24T21:08:20.594ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.554528ms iteration = 46 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
166512026-03-24T21:08:25.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
166522026-03-24T21:08:25.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 137 reason = Timeout
166532026-03-24T21:08:25.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
166542026-03-24T21:08:25.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166552026-03-24T21:08:25.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
166562026-03-24T21:08:25.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166572026-03-24T21:08:25.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166582026-03-24T21:08:25.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166592026-03-24T21:08:25.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
166602026-03-24T21:08:25.541ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166612026-03-24T21:08:25.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
166622026-03-24T21:08:25.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
166632026-03-24T21:08:25.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166642026-03-24T21:08:25.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
166652026-03-24T21:08:25.543ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
166662026-03-24T21:08:25.543ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.965179ms iteration = 137 status = {"config_updated":false}
166672026-03-24T21:08:25.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
166682026-03-24T21:08:25.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
166692026-03-24T21:08:25.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166702026-03-24T21:08:25.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
166712026-03-24T21:08:25.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
166722026-03-24T21:08:25.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
166732026-03-24T21:08:25.562ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
166742026-03-24T21:08:25.562ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
166752026-03-24T21:08:25.562ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
166762026-03-24T21:08:25.562ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.663013ms iteration = 69 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
166772026-03-24T21:08:25.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
166782026-03-24T21:08:25.564ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
166792026-03-24T21:08:25.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
166802026-03-24T21:08:25.565ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.606637ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
166812026-03-24T21:08:28.559ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 728 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = b3f3a26a-9315-4773-8b4f-22aa9042d7d3 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
166822026-03-24T21:08:30.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 138 reason = Timeout
166832026-03-24T21:08:30.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166842026-03-24T21:08:30.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166852026-03-24T21:08:30.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166862026-03-24T21:08:30.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
166872026-03-24T21:08:30.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166882026-03-24T21:08:30.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166892026-03-24T21:08:30.539ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
166902026-03-24T21:08:30.539ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.447374ms iteration = 138 status = {"config_updated":false}
166912026-03-24T21:08:35.522ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 24 reason = Timeout
166922026-03-24T21:08:35.522ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
166932026-03-24T21:08:35.522ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
166942026-03-24T21:08:35.522ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
166952026-03-24T21:08:35.522ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
166962026-03-24T21:08:35.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
166972026-03-24T21:08:35.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
166982026-03-24T21:08:35.525ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 737968421 } })
166992026-03-24T21:08:35.525ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 738734061 } })
167002026-03-24T21:08:35.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
167012026-03-24T21:08:35.526ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 739526592 } })
167022026-03-24T21:08:35.528ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
167032026-03-24T21:08:35.528ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 6.000912ms iteration = 24 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
167042026-03-24T21:08:35.528ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
167052026-03-24T21:08:35.528ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
167062026-03-24T21:08:35.529ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecfc12d7-496f-417d-a7f3-88504509bb3e", "content-length": "26", "date": "Tue, 24 Mar 2026 21:08:35 GMT"} })
167072026-03-24T21:08:35.529ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
167082026-03-24T21:08:35.529ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
167092026-03-24T21:08:35.529ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 742354839 } })
167102026-03-24T21:08:35.531ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 9.458025ms iteration = 24 status = {}
167112026-03-24T21:08:35.531ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 9.562128ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
167122026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
167132026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
167142026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
167152026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
167162026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 36 reason = Timeout
167172026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
167182026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
167192026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
167202026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
167212026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
167222026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 139 reason = Timeout
167232026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 40 reason = Timeout
167242026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167252026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
167262026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 62 reason = Timeout
167272026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
167282026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
167292026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
167302026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
167312026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
167322026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
167332026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
167342026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
167352026-03-24T21:08:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
167362026-03-24T21:08:35.538ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167372026-03-24T21:08:35.539ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
167382026-03-24T21:08:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
167392026-03-24T21:08:35.540ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167402026-03-24T21:08:35.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167412026-03-24T21:08:35.540ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167422026-03-24T21:08:35.541ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167432026-03-24T21:08:35.541ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167442026-03-24T21:08:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
167452026-03-24T21:08:35.541ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
167462026-03-24T21:08:35.542ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167472026-03-24T21:08:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
167482026-03-24T21:08:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
167492026-03-24T21:08:35.545ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
167502026-03-24T21:08:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
167512026-03-24T21:08:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
167522026-03-24T21:08:35.546ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
167532026-03-24T21:08:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
167542026-03-24T21:08:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
167552026-03-24T21:08:35.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
167562026-03-24T21:08:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
167572026-03-24T21:08:35.547ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
167582026-03-24T21:08:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
167592026-03-24T21:08:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
167602026-03-24T21:08:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
167612026-03-24T21:08:35.547ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
167622026-03-24T21:08:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
167632026-03-24T21:08:35.547ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167642026-03-24T21:08:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
167652026-03-24T21:08:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
167662026-03-24T21:08:35.548ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
167672026-03-24T21:08:35.548ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
167682026-03-24T21:08:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
167692026-03-24T21:08:35.548ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
167702026-03-24T21:08:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
167712026-03-24T21:08:35.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
167722026-03-24T21:08:35.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
167732026-03-24T21:08:35.549ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = fae51d14-063d-4c09-aff4-5b856f69c82a original_time_started = 2026-03-24 21:08:06.582692 UTC
167742026-03-24T21:08:35.549ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 15.725774ms iteration = 62 status = {"Loaded":{"collection_id":"fae51d14-063d-4c09-aff4-5b856f69c82a","time_loaded":"2026-03-24T21:08:35.534143112Z","time_started":"2026-03-24T21:08:06.582692Z"}}
167752026-03-24T21:08:35.550ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 763146889 } })
167762026-03-24T21:08:35.550ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 16.577317ms iteration = 24 status = 0
167772026-03-24T21:08:35.550ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167782026-03-24T21:08:35.550ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
167792026-03-24T21:08:35.550ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 16.664759ms iteration = 139 status = {"config_updated":false}
167802026-03-24T21:08:35.550ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
167812026-03-24T21:08:35.551ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 764103605 } })
167822026-03-24T21:08:35.552ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
167832026-03-24T21:08:35.552ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
167842026-03-24T21:08:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
167852026-03-24T21:08:35.552ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
167862026-03-24T21:08:35.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
167872026-03-24T21:08:35.553ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "db32d686-2767-46c1-8dec-23b0931cc1fb", "content-length": "26", "date": "Tue, 24 Mar 2026 21:08:35 GMT"} })
167882026-03-24T21:08:35.553ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
167892026-03-24T21:08:35.553ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
167902026-03-24T21:08:35.555ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 768316289 } })
167912026-03-24T21:08:35.555ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
167922026-03-24T21:08:35.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
167932026-03-24T21:08:35.558ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
167942026-03-24T21:08:35.558ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
167952026-03-24T21:08:35.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
167962026-03-24T21:08:35.559ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1501, tv_nsec: 771796233 } })
167972026-03-24T21:08:35.560ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
167982026-03-24T21:08:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
167992026-03-24T21:08:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
168002026-03-24T21:08:35.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
168012026-03-24T21:08:35.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
168022026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
168032026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
168042026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
168052026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
168062026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
168072026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 36 reason = Timeout
168082026-03-24T21:08:35.561ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
168092026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
168102026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
168112026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
168122026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
168132026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
168142026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
168152026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
168162026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
168172026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
168182026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
168192026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
168202026-03-24T21:08:35.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
168212026-03-24T21:08:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
168222026-03-24T21:08:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
168232026-03-24T21:08:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
168242026-03-24T21:08:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
168252026-03-24T21:08:35.565ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168262026-03-24T21:08:35.567ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
168272026-03-24T21:08:35.567ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
168282026-03-24T21:08:35.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
168292026-03-24T21:08:35.568ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
168302026-03-24T21:08:35.568ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
168312026-03-24T21:08:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
168322026-03-24T21:08:35.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 47 reason = Timeout
168332026-03-24T21:08:35.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 47 reason = Timeout
168342026-03-24T21:08:35.582ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 24 reason = Timeout
168352026-03-24T21:08:35.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
168362026-03-24T21:08:35.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
168372026-03-24T21:08:35.582ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.34\u{b5}s iteration = 24 status = {"disabled":true,"errors":[],"sps":[]}
168382026-03-24T21:08:35.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
168392026-03-24T21:08:35.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
168402026-03-24T21:08:35.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
168412026-03-24T21:08:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
168422026-03-24T21:08:35.586ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168432026-03-24T21:08:35.589ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
168442026-03-24T21:08:35.597ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
168452026-03-24T21:08:35.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
168462026-03-24T21:08:35.603ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
168472026-03-24T21:08:35.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
168482026-03-24T21:08:35.604ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
168492026-03-24T21:08:35.604ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
168502026-03-24T21:08:35.604ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
168512026-03-24T21:08:35.604ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
168522026-03-24T21:08:35.604ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
168532026-03-24T21:08:35.605ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
168542026-03-24T21:08:35.605ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
168552026-03-24T21:08:35.605ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
168562026-03-24T21:08:35.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
168572026-03-24T21:08:35.608ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
168582026-03-24T21:08:35.608ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
168592026-03-24T21:08:35.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168602026-03-24T21:08:35.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
168612026-03-24T21:08:35.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
168622026-03-24T21:08:35.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
168632026-03-24T21:08:35.634ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
168642026-03-24T21:08:35.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
168652026-03-24T21:08:35.639ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
168662026-03-24T21:08:35.639ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
168672026-03-24T21:08:35.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
168682026-03-24T21:08:35.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
168692026-03-24T21:08:35.642ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168702026-03-24T21:08:35.642ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 81.326283ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
168712026-03-24T21:08:35.644ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
168722026-03-24T21:08:35.644ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
168732026-03-24T21:08:35.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
168742026-03-24T21:08:35.646ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168752026-03-24T21:08:35.648ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
168762026-03-24T21:08:35.648ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
168772026-03-24T21:08:35.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
168782026-03-24T21:08:35.649ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
168792026-03-24T21:08:35.649ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
168802026-03-24T21:08:35.649ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
168812026-03-24T21:08:35.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
168822026-03-24T21:08:35.650ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
168832026-03-24T21:08:35.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
168842026-03-24T21:08:35.651ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168852026-03-24T21:08:35.652ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 92.064963ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
168862026-03-24T21:08:35.653ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 94.766405ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
168872026-03-24T21:08:35.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
168882026-03-24T21:08:35.654ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
168892026-03-24T21:08:35.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
168902026-03-24T21:08:35.655ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 93.695176ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
168912026-03-24T21:08:35.656ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
168922026-03-24T21:08:35.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
168932026-03-24T21:08:35.662ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 128.285939ms iteration = 40 status = {"physical_disks_added":0}
168942026-03-24T21:08:35.664ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
168952026-03-24T21:08:35.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
168962026-03-24T21:08:35.666ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
168972026-03-24T21:08:35.666ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 132.838602ms iteration = 70 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
168982026-03-24T21:08:35.668ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168992026-03-24T21:08:35.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
169002026-03-24T21:08:35.671ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
169012026-03-24T21:08:35.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
169022026-03-24T21:08:35.674ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 91.758345ms iteration = 47 status = "NoSitrep"
169032026-03-24T21:08:35.690ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169042026-03-24T21:08:35.692ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
169052026-03-24T21:08:35.697ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
169062026-03-24T21:08:35.697ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
169072026-03-24T21:08:35.697ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 114.717364ms iteration = 47 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
169082026-03-24T21:08:35.707ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 159.08435ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
169092026-03-24T21:08:35.712ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
169102026-03-24T21:08:35.712ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 150.824477ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
169112026-03-24T21:08:35.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
169122026-03-24T21:08:35.713ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
169132026-03-24T21:08:35.713ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
169142026-03-24T21:08:35.713ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
169152026-03-24T21:08:35.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
169162026-03-24T21:08:35.717ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
169172026-03-24T21:08:35.717ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 183.270862ms iteration = 36 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
169182026-03-24T21:08:35.719ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
169192026-03-24T21:08:35.719ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 158.608527ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
169202026-03-24T21:08:35.725ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169212026-03-24T21:08:35.727ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
169222026-03-24T21:08:35.727ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
169232026-03-24T21:08:35.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
169242026-03-24T21:08:35.729ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169252026-03-24T21:08:35.732ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
169262026-03-24T21:08:35.732ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
169272026-03-24T21:08:35.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
169282026-03-24T21:08:35.736ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
169292026-03-24T21:08:35.737ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "76987fb6-426c-484c-810b-b65f291232b5", "content-length": "691", "date": "Tue, 24 Mar 2026 21:08:35 GMT"} })
169302026-03-24T21:08:35.737ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
169312026-03-24T21:08:35.737ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
169322026-03-24T21:08:35.737ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
169332026-03-24T21:08:35.738ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 203.939869ms iteration = 24 status = {}
169342026-03-24T21:08:35.768ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 207.178766ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
169352026-03-24T21:08:35.769ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
169362026-03-24T21:08:35.769ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
169372026-03-24T21:08:35.769ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 208.43117ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
169382026-03-24T21:08:35.770ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
169392026-03-24T21:08:35.770ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
169402026-03-24T21:08:35.777ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e60b67d-1a8e-4888-a75f-70a5199e9a5e", "content-length": "372", "date": "Tue, 24 Mar 2026 21:08:35 GMT"} })
169412026-03-24T21:08:35.777ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169422026-03-24T21:08:35.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169432026-03-24T21:08:35.794ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "44908821-c050-49fa-80a2-966753c5d24c", "date": "Tue, 24 Mar 2026 21:08:35 GMT"} })
169442026-03-24T21:08:35.794ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
169452026-03-24T21:08:35.794ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
169462026-03-24T21:08:35.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
169472026-03-24T21:08:35.835ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
169482026-03-24T21:08:35.835ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
169492026-03-24T21:08:35.835ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f557626d-3f39-43f2-8000-ac1c63a0f680", "date": "Tue, 24 Mar 2026 21:08:35 GMT"} })
169502026-03-24T21:08:35.835ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169512026-03-24T21:08:35.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169522026-03-24T21:08:35.838ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169532026-03-24T21:08:35.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169542026-03-24T21:08:35.841ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169552026-03-24T21:08:35.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169562026-03-24T21:08:35.843ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
169572026-03-24T21:08:35.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
169582026-03-24T21:08:35.845ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
169592026-03-24T21:08:35.845ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
169602026-03-24T21:08:35.846ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "982a649e-d99a-4359-94ba-32f90f8db3e8", "date": "Tue, 24 Mar 2026 21:08:35 GMT"} })
169612026-03-24T21:08:35.846ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 284.633725ms iteration = 36 status = {}
169622026-03-24T21:08:35.846ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
169632026-03-24T21:08:35.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
169642026-03-24T21:08:35.849ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
169652026-03-24T21:08:35.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
169662026-03-24T21:08:35.851ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 316.979448ms iteration = 24 status = {}
169672026-03-24T21:08:38.553ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:47354
169682026-03-24T21:08:38.553ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47354 req_id = 2bdd7ee5-f6ee-4ecb-8fa4-ae4533f7f9ac uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending version = 1.0.0
169692026-03-24T21:08:38.553ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47354 req_id = 2bdd7ee5-f6ee-4ecb-8fa4-ae4533f7f9ac roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
169702026-03-24T21:08:38.553ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47354 req_id = 2bdd7ee5-f6ee-4ecb-8fa4-ae4533f7f9ac resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
169712026-03-24T21:08:38.556ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 3531 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47354 req_id = 2bdd7ee5-f6ee-4ecb-8fa4-ae4533f7f9ac response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?sort_by=id_ascending
169722026-03-24T21:08:38.556ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47354 req_id = 8158ecc0-0613-4524-98e2-d7e44f757d1c uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D version = 1.0.0
169732026-03-24T21:08:38.557ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47354 req_id = 8158ecc0-0613-4524-98e2-d7e44f757d1c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
169742026-03-24T21:08:38.557ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47354 req_id = 8158ecc0-0613-4524-98e2-d7e44f757d1c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
169752026-03-24T21:08:38.559ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 2306 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:47354 req_id = 8158ecc0-0613-4524-98e2-d7e44f757d1c response_code = 200 uri = /metrics/collectors/a309df7c-2a20-4201-8e80-eb9f7e355177/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmOWE2YjI1Zi00ZDlkLTQ1YjgtOWFlNy1jYjRkMWFmMWI1NjEifX0%3D
169762026-03-24T21:08:38.559ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 331 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = dedfd790-17e5-4bd8-8883-d42bdd4da2a7 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
169772026-03-24T21:08:40.534ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 140 reason = Timeout
169782026-03-24T21:08:40.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
169792026-03-24T21:08:40.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
169802026-03-24T21:08:40.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169812026-03-24T21:08:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
169822026-03-24T21:08:40.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
169832026-03-24T21:08:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
169842026-03-24T21:08:40.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
169852026-03-24T21:08:40.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.156843ms iteration = 140 status = {"config_updated":false}
169862026-03-24T21:08:45.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
169872026-03-24T21:08:45.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 141 reason = Timeout
169882026-03-24T21:08:45.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
169892026-03-24T21:08:45.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
169902026-03-24T21:08:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
169912026-03-24T21:08:45.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
169922026-03-24T21:08:45.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169932026-03-24T21:08:45.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169942026-03-24T21:08:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
169952026-03-24T21:08:45.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
169962026-03-24T21:08:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
169972026-03-24T21:08:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
169982026-03-24T21:08:45.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
169992026-03-24T21:08:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170002026-03-24T21:08:45.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
170012026-03-24T21:08:45.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.510163ms iteration = 141 status = {"config_updated":false}
170022026-03-24T21:08:45.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170032026-03-24T21:08:45.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170042026-03-24T21:08:45.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170052026-03-24T21:08:45.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
170062026-03-24T21:08:45.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170072026-03-24T21:08:45.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170082026-03-24T21:08:45.561ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
170092026-03-24T21:08:45.561ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.662295ms iteration = 71 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
170102026-03-24T21:08:45.562ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
170112026-03-24T21:08:45.562ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
170122026-03-24T21:08:45.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
170132026-03-24T21:08:45.602ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
170142026-03-24T21:08:45.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
170152026-03-24T21:08:45.604ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 42.26382ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
170162026-03-24T21:08:48.559ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 369 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = e96fd5cf-a663-4f7b-ac73-ab6b8a9fb4fe response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
170172026-03-24T21:08:50.534ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 63 reason = Timeout
170182026-03-24T21:08:50.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
170192026-03-24T21:08:50.534ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 142 reason = Timeout
170202026-03-24T21:08:50.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170212026-03-24T21:08:50.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
170222026-03-24T21:08:50.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170232026-03-24T21:08:50.536ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170242026-03-24T21:08:50.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170252026-03-24T21:08:50.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
170262026-03-24T21:08:50.540ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
170272026-03-24T21:08:50.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
170282026-03-24T21:08:50.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
170292026-03-24T21:08:50.540ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170302026-03-24T21:08:50.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170312026-03-24T21:08:50.542ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
170322026-03-24T21:08:50.542ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.823711ms iteration = 142 status = {"config_updated":false}
170332026-03-24T21:08:50.542ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = fae51d14-063d-4c09-aff4-5b856f69c82a original_time_started = 2026-03-24 21:08:06.582692 UTC
170342026-03-24T21:08:50.542ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.242563ms iteration = 63 status = {"Loaded":{"collection_id":"fae51d14-063d-4c09-aff4-5b856f69c82a","time_loaded":"2026-03-24T21:08:50.534357576Z","time_started":"2026-03-24T21:08:06.582692Z"}}
170352026-03-24T21:08:50.583ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 48 reason = Timeout
170362026-03-24T21:08:50.583ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 48 reason = Timeout
170372026-03-24T21:08:50.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
170382026-03-24T21:08:50.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
170392026-03-24T21:08:50.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
170402026-03-24T21:08:50.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
170412026-03-24T21:08:50.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
170422026-03-24T21:08:50.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
170432026-03-24T21:08:50.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170442026-03-24T21:08:50.585ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170452026-03-24T21:08:50.587ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
170462026-03-24T21:08:50.587ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
170472026-03-24T21:08:50.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
170482026-03-24T21:08:50.587ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
170492026-03-24T21:08:50.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.697154ms iteration = 48 status = "NoSitrep"
170502026-03-24T21:08:50.590ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
170512026-03-24T21:08:50.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
170522026-03-24T21:08:50.590ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
170532026-03-24T21:08:50.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
170542026-03-24T21:08:50.592ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170552026-03-24T21:08:50.593ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
170562026-03-24T21:08:50.597ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
170572026-03-24T21:08:50.597ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
170582026-03-24T21:08:50.597ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 14.211443ms iteration = 48 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
170592026-03-24T21:08:55.534ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
170602026-03-24T21:08:55.534ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 143 reason = Timeout
170612026-03-24T21:08:55.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170622026-03-24T21:08:55.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170632026-03-24T21:08:55.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170642026-03-24T21:08:55.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170652026-03-24T21:08:55.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170662026-03-24T21:08:55.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170672026-03-24T21:08:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
170682026-03-24T21:08:55.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170692026-03-24T21:08:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170702026-03-24T21:08:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
170712026-03-24T21:08:55.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170722026-03-24T21:08:55.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170732026-03-24T21:08:55.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
170742026-03-24T21:08:55.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.738791ms iteration = 143 status = {"config_updated":false}
170752026-03-24T21:08:55.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170762026-03-24T21:08:55.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170772026-03-24T21:08:55.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170782026-03-24T21:08:55.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
170792026-03-24T21:08:55.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170802026-03-24T21:08:55.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170812026-03-24T21:08:55.558ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
170822026-03-24T21:08:55.558ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.10197ms iteration = 72 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
170832026-03-24T21:08:55.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
170842026-03-24T21:08:55.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
170852026-03-24T21:08:55.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
170862026-03-24T21:08:55.600ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
170872026-03-24T21:08:55.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
170882026-03-24T21:08:55.639ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 77.812988ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
170892026-03-24T21:08:58.559ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 411 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = b3bfb434-7caa-48af-8c49-94fa07a1bf8b response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
170902026-03-24T21:09:00.534ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 144 reason = Timeout
170912026-03-24T21:09:00.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170922026-03-24T21:09:00.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170932026-03-24T21:09:00.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170942026-03-24T21:09:00.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
170952026-03-24T21:09:00.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170962026-03-24T21:09:00.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170972026-03-24T21:09:00.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
170982026-03-24T21:09:00.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.536763ms iteration = 144 status = {"config_updated":false}
170992026-03-24T21:09:05.517ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
171002026-03-24T21:09:05.517ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
171012026-03-24T21:09:05.517ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
171022026-03-24T21:09:05.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
171032026-03-24T21:09:05.518ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
171042026-03-24T21:09:05.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
171052026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171062026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
171072026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
171082026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
171092026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
171102026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
171112026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
171122026-03-24T21:09:05.521ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2026-03-24 20:59:05.521495811 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
171132026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
171142026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
171152026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
171162026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 25 reason = Timeout
171172026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
171182026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
171192026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
171202026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
171212026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
171222026-03-24T21:09:05.521ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
171232026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
171242026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
171252026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
171262026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
171272026-03-24T21:09:05.522ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("93b6e989-1f7d-4884-aec9-ecbc076ed6c4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a38e963c-0406-4fa6-ad7e-c4025bcd6077.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 735019441 } })
171282026-03-24T21:09:05.522ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("27f3a9c5-67c7-41c5-994d-9db513cda557.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2f7ea2ed-fd45-4cd7-9fb1-f48119541118.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("7f3bc00f-0320-479a-a8fb-f67e80ed5929.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 735020951 } })
171292026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
171302026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
171312026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
171322026-03-24T21:09:05.522ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
171332026-03-24T21:09:05.522ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = ExternalNetworking step id = Ensure
171342026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171352026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
171362026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
171372026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.452128ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
171382026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
171392026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.370187ms iteration = 13 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:2::1]:5353"]}
171402026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
171412026-03-24T21:09:05.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171422026-03-24T21:09:05.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
171432026-03-24T21:09:05.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
171442026-03-24T21:09:05.523ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
171452026-03-24T21:09:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "aa360272-8491-4ee4-ae47-ea3176bcb972", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
171462026-03-24T21:09:05.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
171472026-03-24T21:09:05.524ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "7f363960-56f5-487c-a05e-c41899de2a9b", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
171482026-03-24T21:09:05.524ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
171492026-03-24T21:09:05.524ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
171502026-03-24T21:09:05.524ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 2.999021ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
171512026-03-24T21:09:05.524ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2026-03-24 20:59:05.521495811 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
171522026-03-24T21:09:05.524ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.291738ms iteration = 13 status = {"expiration":"2026-03-24T20:59:05.521495811Z","pruned":[]}
171532026-03-24T21:09:05.524ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
171542026-03-24T21:09:05.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171552026-03-24T21:09:05.525ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171562026-03-24T21:09:05.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171572026-03-24T21:09:05.525ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 738583657 } })
171582026-03-24T21:09:05.526ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
171592026-03-24T21:09:05.526ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.648716ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
171602026-03-24T21:09:05.526ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
171612026-03-24T21:09:05.526ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 739395479 } })
171622026-03-24T21:09:05.526ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 739446060 } })
171632026-03-24T21:09:05.526ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171642026-03-24T21:09:05.527ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "40aab3da-4246-46e0-9e88-25f8a690c0dc", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
171652026-03-24T21:09:05.527ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
171662026-03-24T21:09:05.527ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "734cad66-e783-4baf-bef3-8c1207d87426", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
171672026-03-24T21:09:05.527ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
171682026-03-24T21:09:05.527ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "100bd372-826f-47a8-abc9-85be2ca7ab66", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
171692026-03-24T21:09:05.527ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
171702026-03-24T21:09:05.527ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353, [fd00:1122:3344:2::1]:5353] }
171712026-03-24T21:09:05.527ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 5.787786ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
171722026-03-24T21:09:05.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
171732026-03-24T21:09:05.528ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
171742026-03-24T21:09:05.528ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
171752026-03-24T21:09:05.528ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 7.003428ms iteration = 25 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
171762026-03-24T21:09:05.528ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
171772026-03-24T21:09:05.529ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "73ff49f0-1d49-4561-bc61-5142759c4562", "content-length": "26", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} })
171782026-03-24T21:09:05.529ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
171792026-03-24T21:09:05.529ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
171802026-03-24T21:09:05.529ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 742400030 } })
171812026-03-24T21:09:05.529ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
171822026-03-24T21:09:05.529ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
171832026-03-24T21:09:05.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
171842026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
171852026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171862026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 11.762557ms iteration = 25 status = {}
171872026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171882026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 37 reason = Timeout
171892026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
171902026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
171912026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 31 reason = Timeout
171922026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
171932026-03-24T21:09:05.533ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
171942026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
171952026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
171962026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
171972026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
171982026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
171992026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 41 reason = Timeout
172002026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 64 reason = Timeout
172012026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
172022026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
172032026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
172042026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
172052026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
172062026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
172072026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
172082026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
172092026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 145 reason = Timeout
172102026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
172112026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
172122026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
172132026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
172142026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
172152026-03-24T21:09:05.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
172162026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
172172026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
172182026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
172192026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
172202026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
172212026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
172222026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
172232026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
172242026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
172252026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
172262026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
172272026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
172282026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
172292026-03-24T21:09:05.534ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
172302026-03-24T21:09:05.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
172312026-03-24T21:09:05.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
172322026-03-24T21:09:05.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172332026-03-24T21:09:05.536ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172342026-03-24T21:09:05.537ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172352026-03-24T21:09:05.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172362026-03-24T21:09:05.537ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172372026-03-24T21:09:05.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172382026-03-24T21:09:05.538ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172392026-03-24T21:09:05.539ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
172402026-03-24T21:09:05.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
172412026-03-24T21:09:05.540ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172422026-03-24T21:09:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
172432026-03-24T21:09:05.542ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
172442026-03-24T21:09:05.542ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172452026-03-24T21:09:05.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
172462026-03-24T21:09:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
172472026-03-24T21:09:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
172482026-03-24T21:09:05.543ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
172492026-03-24T21:09:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
172502026-03-24T21:09:05.543ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
172512026-03-24T21:09:05.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
172522026-03-24T21:09:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
172532026-03-24T21:09:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
172542026-03-24T21:09:05.544ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
172552026-03-24T21:09:05.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
172562026-03-24T21:09:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
172572026-03-24T21:09:05.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
172582026-03-24T21:09:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
172592026-03-24T21:09:05.545ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
172602026-03-24T21:09:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
172612026-03-24T21:09:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
172622026-03-24T21:09:05.545ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
172632026-03-24T21:09:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
172642026-03-24T21:09:05.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
172652026-03-24T21:09:05.547ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
172662026-03-24T21:09:05.547ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
172672026-03-24T21:09:05.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
172682026-03-24T21:09:05.548ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = fae51d14-063d-4c09-aff4-5b856f69c82a original_time_started = 2026-03-24 21:08:06.582692 UTC
172692026-03-24T21:09:05.548ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 14.962673ms iteration = 64 status = {"Loaded":{"collection_id":"fae51d14-063d-4c09-aff4-5b856f69c82a","time_loaded":"2026-03-24T21:09:05.533474264Z","time_started":"2026-03-24T21:08:06.582692Z"}}
172702026-03-24T21:09:05.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
172712026-03-24T21:09:05.548ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
172722026-03-24T21:09:05.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
172732026-03-24T21:09:05.550ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
172742026-03-24T21:09:05.551ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172752026-03-24T21:09:05.551ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
172762026-03-24T21:09:05.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
172772026-03-24T21:09:05.553ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 20.179274ms iteration = 25 status = 0
172782026-03-24T21:09:05.553ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172792026-03-24T21:09:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
172802026-03-24T21:09:05.554ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
172812026-03-24T21:09:05.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
172822026-03-24T21:09:05.556ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
172832026-03-24T21:09:05.556ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
172842026-03-24T21:09:05.556ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
172852026-03-24T21:09:05.556ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
172862026-03-24T21:09:05.556ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
172872026-03-24T21:09:05.556ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 22.714483ms iteration = 145 status = {"config_updated":false}
172882026-03-24T21:09:05.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
172892026-03-24T21:09:05.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
172902026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
172912026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
172922026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
172932026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 29 reason = Timeout
172942026-03-24T21:09:05.558ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
172952026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
172962026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
172972026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 19.12\u{b5}s iteration = 29 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
172982026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
172992026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
173002026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
173012026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
173022026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
173032026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
173042026-03-24T21:09:05.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
173052026-03-24T21:09:05.560ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.605935ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
173062026-03-24T21:09:05.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
173072026-03-24T21:09:05.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
173082026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
173092026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
173102026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
173112026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
173122026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
173132026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
173142026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
173152026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
173162026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
173172026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
173182026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
173192026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
173202026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
173212026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 37 reason = Timeout
173222026-03-24T21:09:05.561ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
173232026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
173242026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
173252026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
173262026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
173272026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
173282026-03-24T21:09:05.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
173292026-03-24T21:09:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
173302026-03-24T21:09:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
173312026-03-24T21:09:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
173322026-03-24T21:09:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
173332026-03-24T21:09:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
173342026-03-24T21:09:05.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
173352026-03-24T21:09:05.562ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
173362026-03-24T21:09:05.562ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 3.986647ms iteration = 13 status = {"by_rx":{},"error":null}
173372026-03-24T21:09:05.563ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 776710605 } })
173382026-03-24T21:09:05.564ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
173392026-03-24T21:09:05.564ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 777585379 } })
173402026-03-24T21:09:05.566ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
173412026-03-24T21:09:05.566ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
173422026-03-24T21:09:05.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
173432026-03-24T21:09:05.566ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
173442026-03-24T21:09:05.566ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 5.628192ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
173452026-03-24T21:09:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
173462026-03-24T21:09:05.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
173472026-03-24T21:09:05.567ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "da18eb62-46b7-40b3-82f3-ac1c5fd82e92", "content-length": "26", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} })
173482026-03-24T21:09:05.567ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
173492026-03-24T21:09:05.567ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
173502026-03-24T21:09:05.567ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 10931512-f188-4be0-ab88-10601b6a8268 (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 } zone_id = 81b21d3e-108a-4e68-a045-5b9ba92ee7cf zone_kind = nexus
173512026-03-24T21:09:05.568ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173522026-03-24T21:09:05.569ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
173532026-03-24T21:09:05.569ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 782363768 } })
173542026-03-24T21:09:05.569ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 8.279564ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
173552026-03-24T21:09:05.571ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
173562026-03-24T21:09:05.571ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
173572026-03-24T21:09:05.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
173582026-03-24T21:09:05.572ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 99e04159-2e39-4678-98ae-aac44d89a95c (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 } zone_id = 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 zone_kind = external_dns
173592026-03-24T21:09:05.572ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
173602026-03-24T21:09:05.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
173612026-03-24T21:09:05.573ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 785850552 } })
173622026-03-24T21:09:05.573ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
173632026-03-24T21:09:05.574ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
173642026-03-24T21:09:05.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
173652026-03-24T21:09:05.576ZDEBGnexus (BackgroundTasks): activating background_task = trust_quorum_manager iteration = 13 reason = Timeout
173662026-03-24T21:09:05.576ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 13 reason = Timeout
173672026-03-24T21:09:05.576ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
173682026-03-24T21:09:05.576ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
173692026-03-24T21:09:05.576ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
173702026-03-24T21:09:05.576ZDEBGnexus (BackgroundTasks): activating background_task = attached_subnet_manager iteration = 13 reason = Timeout
173712026-03-24T21:09:05.576ZINFOnexus (BackgroundTasks): starting attached subnet manager work background_task = attached_subnet_manager file = nexus/src/app/background/tasks/attached_subnets.rs:374
173722026-03-24T21:09:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
173732026-03-24T21:09:05.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
173742026-03-24T21:09:05.577ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1531, tv_nsec: 790433515 } })
173752026-03-24T21:09:05.577ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
173762026-03-24T21:09:05.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
173772026-03-24T21:09:05.579ZDEBGnexus (BackgroundTasks): roles background_task = attached_subnet_manager roles = RoleSet { roles: {} }
173782026-03-24T21:09:05.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = attached_subnet_manager resource = Database result = Ok(())
173792026-03-24T21:09:05.580ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173802026-03-24T21:09:05.583ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 25 reason = Timeout
173812026-03-24T21:09:05.583ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 16.8\u{b5}s iteration = 25 status = {"disabled":true,"errors":[],"sps":[]}
173822026-03-24T21:09:05.583ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 49 reason = Timeout
173832026-03-24T21:09:05.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
173842026-03-24T21:09:05.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
173852026-03-24T21:09:05.584ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 49 reason = Timeout
173862026-03-24T21:09:05.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
173872026-03-24T21:09:05.584ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
173882026-03-24T21:09:05.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
173892026-03-24T21:09:05.584ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
173902026-03-24T21:09:05.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
173912026-03-24T21:09:05.584ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
173922026-03-24T21:09:05.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
173932026-03-24T21:09:05.585ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 33dedc91-2fbf-4d60-a250-667569ca6c52 (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 } zone_id = a38e963c-0406-4fa6-ad7e-c4025bcd6077 zone_kind = external_dns
173942026-03-24T21:09:05.589ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 31.279433ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"242.0.0","globs_reprocessed":{},"no_receivers":[]}
173952026-03-24T21:09:05.590ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
173962026-03-24T21:09:05.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173972026-03-24T21:09:05.607ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: f723520c-8acb-4131-8d8e-eb41ab4882a9 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 } zone_id = da73ac6f-331d-476f-a388-d3f6ef4cc9cb zone_kind = nexus
173982026-03-24T21:09:05.609ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
173992026-03-24T21:09:05.609ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
174002026-03-24T21:09:05.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
174012026-03-24T21:09:05.612ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Snat(OmicronZoneExternalSnatIp { id: 626966fe-1bee-4313-9d9d-053f8b600da0 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: dbc1027e-4275-4ba2-b04c-a77a49e8291f, kind: Service { id: ea49db54-0bfa-4e3d-bc93-404c287645e6 }, name: Name("ntp-ea49db54-0bfa-4e3d-bc93-404c287645e6"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 211, 56])), vni: Vni(100), primary: true, slot: 0 } zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 zone_kind = boundary_ntp
174022026-03-24T21:09:05.612ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
174032026-03-24T21:09:05.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
174042026-03-24T21:09:05.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
174052026-03-24T21:09:05.613ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:282 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
174062026-03-24T21:09:05.614ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
174072026-03-24T21:09:05.614ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
174082026-03-24T21:09:05.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
174092026-03-24T21:09:05.616ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174102026-03-24T21:09:05.617ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:349 ip = Floating(OmicronZoneExternalFloatingIp { id: 7c0aa389-6a9f-4c2a-a630-0ac976189e52 (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 } zone_id = efeeb6e2-455c-4e0b-a539-22214aaf7506 zone_kind = nexus
174112026-03-24T21:09:05.619ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = SledList step id = Fetch
174122026-03-24T21:09:05.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
174132026-03-24T21:09:05.619ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174142026-03-24T21:09:05.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
174152026-03-24T21:09:05.621ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
174162026-03-24T21:09:05.621ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
174172026-03-24T21:09:05.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
174182026-03-24T21:09:05.623ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Fleet result = Ok(())
174192026-03-24T21:09:05.623ZDEBGnexus (BackgroundTasks): roles background_task = trust_quorum_manager bgtask = TrustQuorumManager roles = RoleSet { roles: {} }
174202026-03-24T21:09:05.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = trust_quorum_manager bgtask = TrustQuorumManager resource = Database result = Ok(())
174212026-03-24T21:09:05.642ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
174222026-03-24T21:09:05.642ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
174232026-03-24T21:09:05.642ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 108.970989ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
174242026-03-24T21:09:05.643ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
174252026-03-24T21:09:05.643ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
174262026-03-24T21:09:05.643ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
174272026-03-24T21:09:05.643ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
174282026-03-24T21:09:05.644ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174292026-03-24T21:09:05.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
174302026-03-24T21:09:05.644ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
174312026-03-24T21:09:05.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
174322026-03-24T21:09:05.645ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174332026-03-24T21:09:05.646ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
174342026-03-24T21:09:05.646ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
174352026-03-24T21:09:05.646ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
174362026-03-24T21:09:05.646ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
174372026-03-24T21:09:05.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
174382026-03-24T21:09:05.647ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174392026-03-24T21:09:05.648ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
174402026-03-24T21:09:05.648ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
174412026-03-24T21:09:05.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
174422026-03-24T21:09:05.649ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174432026-03-24T21:09:05.649ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
174442026-03-24T21:09:05.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
174452026-03-24T21:09:05.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
174462026-03-24T21:09:05.651ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
174472026-03-24T21:09:05.651ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
174482026-03-24T21:09:05.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
174492026-03-24T21:09:05.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
174502026-03-24T21:09:05.653ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
174512026-03-24T21:09:05.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
174522026-03-24T21:09:05.653ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174532026-03-24T21:09:05.657ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174542026-03-24T21:09:05.660ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174552026-03-24T21:09:05.660ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174562026-03-24T21:09:05.660ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
174572026-03-24T21:09:05.660ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
174582026-03-24T21:09:05.661ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174592026-03-24T21:09:05.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
174602026-03-24T21:09:05.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
174612026-03-24T21:09:05.662ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
174622026-03-24T21:09:05.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
174632026-03-24T21:09:05.664ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174642026-03-24T21:09:05.665ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
174652026-03-24T21:09:05.665ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
174662026-03-24T21:09:05.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
174672026-03-24T21:09:05.665ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
174682026-03-24T21:09:05.666ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174692026-03-24T21:09:05.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174702026-03-24T21:09:05.667ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174712026-03-24T21:09:05.667ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
174722026-03-24T21:09:05.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
174732026-03-24T21:09:05.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
174742026-03-24T21:09:05.668ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174752026-03-24T21:09:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
174762026-03-24T21:09:05.670ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 87.448218ms iteration = 49 status = "NoSitrep"
174772026-03-24T21:09:05.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
174782026-03-24T21:09:05.670ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
174792026-03-24T21:09:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
174802026-03-24T21:09:05.671ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174812026-03-24T21:09:05.674ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
174822026-03-24T21:09:05.674ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
174832026-03-24T21:09:05.674ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
174842026-03-24T21:09:05.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
174852026-03-24T21:09:05.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
174862026-03-24T21:09:05.675ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 142.278516ms iteration = 6 status = {"nsuccess":5}
174872026-03-24T21:09:05.675ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
174882026-03-24T21:09:05.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
174892026-03-24T21:09:05.677ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
174902026-03-24T21:09:05.677ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
174912026-03-24T21:09:05.677ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
174922026-03-24T21:09:05.677ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
174932026-03-24T21:09:05.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
174942026-03-24T21:09:05.678ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a0579c2-6683-4dde-b548-cc373d5ce96f", "content-length": "372", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} })
174952026-03-24T21:09:05.678ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
174962026-03-24T21:09:05.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
174972026-03-24T21:09:05.679ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
174982026-03-24T21:09:05.679ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
174992026-03-24T21:09:05.679ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 123.234813ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
175002026-03-24T21:09:05.681ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
175012026-03-24T21:09:05.681ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
175022026-03-24T21:09:05.681ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
175032026-03-24T21:09:05.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
175042026-03-24T21:09:05.682ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175052026-03-24T21:09:05.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175062026-03-24T21:09:05.685ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
175072026-03-24T21:09:05.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
175082026-03-24T21:09:05.686ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175092026-03-24T21:09:05.686ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
175102026-03-24T21:09:05.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
175112026-03-24T21:09:05.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
175122026-03-24T21:09:05.687ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
175132026-03-24T21:09:05.687ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
175142026-03-24T21:09:05.687ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
175152026-03-24T21:09:05.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175162026-03-24T21:09:05.687ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
175172026-03-24T21:09:05.687ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
175182026-03-24T21:09:05.687ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
175192026-03-24T21:09:05.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175202026-03-24T21:09:05.687ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
175212026-03-24T21:09:05.687ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
175222026-03-24T21:09:05.687ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
175232026-03-24T21:09:05.687ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
175242026-03-24T21:09:05.687ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
175252026-03-24T21:09:05.687ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
175262026-03-24T21:09:05.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
175272026-03-24T21:09:05.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
175282026-03-24T21:09:05.688ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
175292026-03-24T21:09:05.690ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175302026-03-24T21:09:05.695ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 148.261548ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
175312026-03-24T21:09:05.696ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
175322026-03-24T21:09:05.696ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
175332026-03-24T21:09:05.696ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 112.695099ms iteration = 49 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
175342026-03-24T21:09:05.696ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
175352026-03-24T21:09:05.696ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
175362026-03-24T21:09:05.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
175372026-03-24T21:09:05.705ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
175382026-03-24T21:09:05.705ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 171.828773ms iteration = 73 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
175392026-03-24T21:09:05.706ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
175402026-03-24T21:09:05.706ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 144.876776ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
175412026-03-24T21:09:05.707ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 148.945737ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
175422026-03-24T21:09:05.710ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
175432026-03-24T21:09:05.710ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
175442026-03-24T21:09:05.710ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
175452026-03-24T21:09:05.710ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
175462026-03-24T21:09:05.710ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2026-03-24 20:57:05.047625 UTC dns_group = internal generation = 1
175472026-03-24T21:09:05.710ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 193.852658ms iteration = 13 status = {"generation":1}
175482026-03-24T21:09:05.739ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 177.2875ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
175492026-03-24T21:09:05.751ZDEBGnexus (BackgroundTasks): listed attached subnets background_task = attached_subnet_manager n_subnets = 0
175502026-03-24T21:09:05.752ZDEBGnexus (BackgroundTasks): activation complete background_task = attached_subnet_manager elapsed = 175.294087ms iteration = 13 status = {"db_error":null,"dendrite":{"switch0":{"errors":[],"n_subnets_added":0,"n_subnets_removed":0,"n_total_subnets":0}},"sled":{}}
175512026-03-24T21:09:05.754ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 192.386208ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
175522026-03-24T21:09:05.764ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 202.610444ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
175532026-03-24T21:09:05.765ZINFOnexus (BackgroundTasks): Loaded 0 active trust quorum configurations from database background_task = trust_quorum_manager bgtask = TrustQuorumManager file = nexus/src/app/background/tasks/trust_quorum.rs:83
175542026-03-24T21:09:05.765ZDEBGnexus (BackgroundTasks): activation complete background_task = trust_quorum_manager elapsed = 188.666507ms iteration = 13 status = {"PerRackStatus":{"errors":[],"statuses":[]}}
175552026-03-24T21:09:05.778ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
175562026-03-24T21:09:05.778ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
175572026-03-24T21:09:05.778ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
175582026-03-24T21:09:05.778ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
175592026-03-24T21:09:05.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
175602026-03-24T21:09:05.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
175612026-03-24T21:09:05.780ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175622026-03-24T21:09:05.782ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
175632026-03-24T21:09:05.782ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
175642026-03-24T21:09:05.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
175652026-03-24T21:09:05.784ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
175662026-03-24T21:09:05.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
175672026-03-24T21:09:05.786ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175682026-03-24T21:09:05.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
175692026-03-24T21:09:05.790ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
175702026-03-24T21:09:05.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
175712026-03-24T21:09:05.793ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
175722026-03-24T21:09:05.793ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 260.327349ms iteration = 37 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
175732026-03-24T21:09:05.806ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
175742026-03-24T21:09:05.806ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
175752026-03-24T21:09:05.806ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
175762026-03-24T21:09:05.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
175772026-03-24T21:09:05.808ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
175782026-03-24T21:09:05.808ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
175792026-03-24T21:09:05.808ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5efe1701-31de-43dc-9f17-df09b7dc9f2f", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} })
175802026-03-24T21:09:05.808ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
175812026-03-24T21:09:05.808ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175822026-03-24T21:09:05.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
175832026-03-24T21:09:05.812ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
175842026-03-24T21:09:05.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
175852026-03-24T21:09:05.814ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
175862026-03-24T21:09:05.814ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
175872026-03-24T21:09:05.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
175882026-03-24T21:09:05.815ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
175892026-03-24T21:09:05.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
175902026-03-24T21:09:05.816ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
175912026-03-24T21:09:05.817ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
175922026-03-24T21:09:05.817ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2026-03-24 20:57:07.125641 UTC dns_group = external generation = 2
175932026-03-24T21:09:05.817ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 295.539579ms iteration = 13 status = {"generation":2}
175942026-03-24T21:09:05.819ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
175952026-03-24T21:09:05.819ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
175962026-03-24T21:09:05.825ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e4d0d124-10d3-4f4a-add7-99531157ca1d", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} })
175972026-03-24T21:09:05.825ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 264.165504ms iteration = 37 status = {}
175982026-03-24T21:09:05.828ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
175992026-03-24T21:09:05.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
176002026-03-24T21:09:05.846ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
176012026-03-24T21:09:05.846ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
176022026-03-24T21:09:05.846ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 285.215481ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
176032026-03-24T21:09:05.847ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 313.832213ms iteration = 41 status = {"physical_disks_added":0}
176042026-03-24T21:09:05.848ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
176052026-03-24T21:09:05.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
176062026-03-24T21:09:05.852ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2026-03-24T20:56:59.957750Z, time_modified: 2026-03-24T20:56:59.957750Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, db_silo: Silo { identity: SiloIdentity { id: 9cbddfea-14c3-4878-940f-9c5bf27dec6d, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2026-03-24T20:57:07.044378Z, time_modified: 2026-03-24T20:57:07.044378Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096" }] }) }
176072026-03-24T21:09:05.852ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 330.499782ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]}},"default_endpoint":{"silo_id":"9cbddfea-14c3-4878-940f-9c5bf27dec6d","tls_certs":["7884123e3723c925bfdc886618ddc301f23ed82ef35dca9cdfd854326ef76096"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
176082026-03-24T21:09:05.857ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = DeployNexusRecords step id = Ensure
176092026-03-24T21:09:05.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
176102026-03-24T21:09:05.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
176112026-03-24T21:09:05.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176122026-03-24T21:09:05.862ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
176132026-03-24T21:09:05.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
176142026-03-24T21:09:05.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
176152026-03-24T21:09:05.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
176162026-03-24T21:09:05.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
176172026-03-24T21:09:05.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176182026-03-24T21:09:05.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
176192026-03-24T21:09:05.879ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
176202026-03-24T21:09:05.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
176212026-03-24T21:09:05.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176222026-03-24T21:09:05.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176232026-03-24T21:09:05.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176242026-03-24T21:09:05.885ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
176252026-03-24T21:09:05.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176262026-03-24T21:09:05.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176272026-03-24T21:09:05.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176282026-03-24T21:09:05.890ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
176292026-03-24T21:09:05.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176302026-03-24T21:09:05.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176312026-03-24T21:09:05.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176322026-03-24T21:09:05.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176332026-03-24T21:09:05.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176342026-03-24T21:09:05.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176352026-03-24T21:09:05.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176362026-03-24T21:09:05.905ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9b646ab3-c46e-40b2-a107-8ea2a55c5133", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} })
176372026-03-24T21:09:05.905ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
176382026-03-24T21:09:05.905ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
176392026-03-24T21:09:05.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
176402026-03-24T21:09:05.906ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
176412026-03-24T21:09:05.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176422026-03-24T21:09:05.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176432026-03-24T21:09:05.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176442026-03-24T21:09:05.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176452026-03-24T21:09:05.916ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176462026-03-24T21:09:05.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176472026-03-24T21:09:05.919ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
176482026-03-24T21:09:05.920ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
176492026-03-24T21:09:05.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
176502026-03-24T21:09:05.922ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
176512026-03-24T21:09:05.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176522026-03-24T21:09:05.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176532026-03-24T21:09:05.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176542026-03-24T21:09:05.928ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
176552026-03-24T21:09:05.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176562026-03-24T21:09:05.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176572026-03-24T21:09:05.931ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
176582026-03-24T21:09:05.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176592026-03-24T21:09:05.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176602026-03-24T21:09:05.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176612026-03-24T21:09:05.934ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = SledAgent step id = Ensure
176622026-03-24T21:09:05.935ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b4293d4f-3308-4dfa-b261-a103215913e4 uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
176632026-03-24T21:09:05.936ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
176642026-03-24T21:09:05.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176652026-03-24T21:09:05.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176662026-03-24T21:09:05.937ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "8af154d8-c17a-428a-96e9-aa597d5cb5d5", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} })
176672026-03-24T21:09:05.937ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
176682026-03-24T21:09:05.937ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
176692026-03-24T21:09:05.937ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 360.680486ms iteration = 13 status = {"errors":[],"probes_by_sled":{"b4293d4f-3308-4dfa-b261-a103215913e4":0}}
176702026-03-24T21:09:05.937ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 38 reason = Dependency
176712026-03-24T21:09:05.937ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
176722026-03-24T21:09:05.937ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
176732026-03-24T21:09:05.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
176742026-03-24T21:09:05.939ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176752026-03-24T21:09:05.939ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "b125fd8b-7914-4f80-a127-c13c7348fde0", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} }) sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
176762026-03-24T21:09:05.939ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = FirewallRules step id = Ensure
176772026-03-24T21:09:05.939ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
176782026-03-24T21:09:05.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176792026-03-24T21:09:05.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
176802026-03-24T21:09:05.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176812026-03-24T21:09:05.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176822026-03-24T21:09:05.941ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
176832026-03-24T21:09:05.941ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
176842026-03-24T21:09:05.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
176852026-03-24T21:09:05.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
176862026-03-24T21:09:05.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
176872026-03-24T21:09:05.943ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176882026-03-24T21:09:05.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
176892026-03-24T21:09:05.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
176902026-03-24T21:09:05.945ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
176912026-03-24T21:09:05.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176922026-03-24T21:09:05.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176932026-03-24T21:09:05.946ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
176942026-03-24T21:09:05.946ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
176952026-03-24T21:09:05.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
176962026-03-24T21:09:05.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176972026-03-24T21:09:05.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
176982026-03-24T21:09:05.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176992026-03-24T21:09:05.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177002026-03-24T21:09:05.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177012026-03-24T21:09:05.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177022026-03-24T21:09:05.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177032026-03-24T21:09:05.950ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
177042026-03-24T21:09:05.950ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
177052026-03-24T21:09:05.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177062026-03-24T21:09:05.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177072026-03-24T21:09:05.952ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
177082026-03-24T21:09:05.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177092026-03-24T21:09:05.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177102026-03-24T21:09:05.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177112026-03-24T21:09:05.957ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "21ded828-90f3-41f2-8cce-eaf50963c6e9", "content-length": "372", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} })
177122026-03-24T21:09:05.957ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
177132026-03-24T21:09:05.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
177142026-03-24T21:09:05.959ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
177152026-03-24T21:09:05.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177162026-03-24T21:09:05.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177172026-03-24T21:09:05.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177182026-03-24T21:09:05.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177192026-03-24T21:09:05.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177202026-03-24T21:09:05.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177212026-03-24T21:09:05.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177222026-03-24T21:09:05.969ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
177232026-03-24T21:09:05.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177242026-03-24T21:09:05.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177252026-03-24T21:09:05.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177262026-03-24T21:09:05.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177272026-03-24T21:09:05.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177282026-03-24T21:09:05.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177292026-03-24T21:09:05.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177302026-03-24T21:09:05.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177312026-03-24T21:09:05.976ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177322026-03-24T21:09:05.976ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
177332026-03-24T21:09:05.976ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
177342026-03-24T21:09:05.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177352026-03-24T21:09:05.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177362026-03-24T21:09:05.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177372026-03-24T21:09:05.979ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6799d70e-87e2-4172-85c8-9002a059f68c", "date": "Tue, 24 Mar 2026 21:09:05 GMT"} })
177382026-03-24T21:09:05.979ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
177392026-03-24T21:09:05.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
177402026-03-24T21:09:05.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177412026-03-24T21:09:05.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177422026-03-24T21:09:05.984ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
177432026-03-24T21:09:05.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177442026-03-24T21:09:05.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
177452026-03-24T21:09:05.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177462026-03-24T21:09:05.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177472026-03-24T21:09:05.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177482026-03-24T21:09:05.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177492026-03-24T21:09:05.989ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
177502026-03-24T21:09:05.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
177512026-03-24T21:09:05.991ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
177522026-03-24T21:09:05.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
177532026-03-24T21:09:05.993ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
177542026-03-24T21:09:05.993ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
177552026-03-24T21:09:05.993ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
177562026-03-24T21:09:05.993ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177572026-03-24T21:09:05.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177582026-03-24T21:09:05.995ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
177592026-03-24T21:09:05.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
177602026-03-24T21:09:05.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177612026-03-24T21:09:05.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177622026-03-24T21:09:05.997ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
177632026-03-24T21:09:05.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
177642026-03-24T21:09:05.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177652026-03-24T21:09:05.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177662026-03-24T21:09:06.001ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 467.684111ms iteration = 25 status = {}
177672026-03-24T21:09:06.001ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d7fc6e56-c2e7-472d-8672-75c650c005f8", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
177682026-03-24T21:09:06.001ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 64.028917ms iteration = 38 status = {}
177692026-03-24T21:09:06.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177702026-03-24T21:09:06.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177712026-03-24T21:09:06.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177722026-03-24T21:09:06.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177732026-03-24T21:09:06.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177742026-03-24T21:09:06.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177752026-03-24T21:09:06.009ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:118 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
177762026-03-24T21:09:06.009ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 448.32685ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
177772026-03-24T21:09:06.009ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177782026-03-24T21:09:06.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177792026-03-24T21:09:06.011ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177802026-03-24T21:09:06.016ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
177812026-03-24T21:09:06.016ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177822026-03-24T21:09:06.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177832026-03-24T21:09:06.018ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177842026-03-24T21:09:06.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177852026-03-24T21:09:06.019ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177862026-03-24T21:09:06.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177872026-03-24T21:09:06.021ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177882026-03-24T21:09:06.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177892026-03-24T21:09:06.026ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
177902026-03-24T21:09:06.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177912026-03-24T21:09:06.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177922026-03-24T21:09:06.027ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
177932026-03-24T21:09:06.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177942026-03-24T21:09:06.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177952026-03-24T21:09:06.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177962026-03-24T21:09:06.030ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
177972026-03-24T21:09:06.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177982026-03-24T21:09:06.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177992026-03-24T21:09:06.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178002026-03-24T21:09:06.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178012026-03-24T21:09:06.033ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178022026-03-24T21:09:06.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178032026-03-24T21:09:06.033ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178042026-03-24T21:09:06.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178052026-03-24T21:09:06.034ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178062026-03-24T21:09:06.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178072026-03-24T21:09:06.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178082026-03-24T21:09:06.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178092026-03-24T21:09:06.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178102026-03-24T21:09:06.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178112026-03-24T21:09:06.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178122026-03-24T21:09:06.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178132026-03-24T21:09:06.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178142026-03-24T21:09:06.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178152026-03-24T21:09:06.041ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
178162026-03-24T21:09:06.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178172026-03-24T21:09:06.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178182026-03-24T21:09:06.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178192026-03-24T21:09:06.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178202026-03-24T21:09:06.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178212026-03-24T21:09:06.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178222026-03-24T21:09:06.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178232026-03-24T21:09:06.046ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
178242026-03-24T21:09:06.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178252026-03-24T21:09:06.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178262026-03-24T21:09:06.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178272026-03-24T21:09:06.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178282026-03-24T21:09:06.050ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178292026-03-24T21:09:06.050ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
178302026-03-24T21:09:06.050ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178312026-03-24T21:09:06.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178322026-03-24T21:09:06.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178332026-03-24T21:09:06.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178342026-03-24T21:09:06.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178352026-03-24T21:09:06.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178362026-03-24T21:09:06.053ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178372026-03-24T21:09:06.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178382026-03-24T21:09:06.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178392026-03-24T21:09:06.055ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
178402026-03-24T21:09:06.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178412026-03-24T21:09:06.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178422026-03-24T21:09:06.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178432026-03-24T21:09:06.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178442026-03-24T21:09:06.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178452026-03-24T21:09:06.061ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
178462026-03-24T21:09:06.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178472026-03-24T21:09:06.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178482026-03-24T21:09:06.064ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
178492026-03-24T21:09:06.064ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
178502026-03-24T21:09:06.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
178512026-03-24T21:09:06.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178522026-03-24T21:09:06.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178532026-03-24T21:09:06.066ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178542026-03-24T21:09:06.067ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
178552026-03-24T21:09:06.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178562026-03-24T21:09:06.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178572026-03-24T21:09:06.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178582026-03-24T21:09:06.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178592026-03-24T21:09:06.068ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
178602026-03-24T21:09:06.068ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
178612026-03-24T21:09:06.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
178622026-03-24T21:09:06.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178632026-03-24T21:09:06.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178642026-03-24T21:09:06.069ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178652026-03-24T21:09:06.071ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
178662026-03-24T21:09:06.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178672026-03-24T21:09:06.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178682026-03-24T21:09:06.073ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
178692026-03-24T21:09:06.073ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
178702026-03-24T21:09:06.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
178712026-03-24T21:09:06.073ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
178722026-03-24T21:09:06.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178732026-03-24T21:09:06.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178742026-03-24T21:09:06.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178752026-03-24T21:09:06.076ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
178762026-03-24T21:09:06.076ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 210993e6-e5d3-4adc-bf37-16574823b363, kind: Service { id: 93b6e989-1f7d-4884-aec9-ecbc076ed6c4 }, name: Name("external-dns-93b6e989-1f7d-4884-aec9-ecbc076ed6c4"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 184, 106])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 98d617a0-1253-4bf7-aee3-4170a238ab13, kind: Service { id: a38e963c-0406-4fa6-ad7e-c4025bcd6077 }, name: Name("external-dns-a38e963c-0406-4fa6-ad7e-c4025bcd6077"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 228, 54])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 08ea2928-860f-43c3-b93e-8fd1dbb64238, kind: Service { id: efeeb6e2-455c-4e0b-a539-22214aaf7506 }, name: Name("nexus-efeeb6e2-455c-4e0b-a539-22214aaf7506"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 176, 249])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 20e89fb0-8289-45aa-b7e2-94d63ef84d75, kind: Service { id: 81b21d3e-108a-4e68-a045-5b9ba92ee7cf }, name: Name("nexus-81b21d3e-108a-4e68-a045-5b9ba92ee7cf"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 244, 119])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: da79c301-24a0-4c18-9384-e09f71200807, kind: Service { id: da73ac6f-331d-476f-a388-d3f6ef4cc9cb }, name: Name("nexus-da73ac6f-331d-476f-a388-d3f6ef4cc9cb"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 214, 166])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
178772026-03-24T21:09:06.076ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
178782026-03-24T21:09:06.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178792026-03-24T21:09:06.077ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
178802026-03-24T21:09:06.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178812026-03-24T21:09:06.078ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0c6a8e8-cd77-445f-ab7d-737c5e97a6fd", "content-length": "691", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
178822026-03-24T21:09:06.078ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
178832026-03-24T21:09:06.078ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
178842026-03-24T21:09:06.078ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
178852026-03-24T21:09:06.078ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 544.817211ms iteration = 25 status = {}
178862026-03-24T21:09:06.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
178872026-03-24T21:09:06.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178882026-03-24T21:09:06.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178892026-03-24T21:09:06.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178902026-03-24T21:09:06.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178912026-03-24T21:09:06.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178922026-03-24T21:09:06.087ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
178932026-03-24T21:09:06.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178942026-03-24T21:09:06.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178952026-03-24T21:09:06.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178962026-03-24T21:09:06.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178972026-03-24T21:09:06.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178982026-03-24T21:09:06.092ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
178992026-03-24T21:09:06.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179002026-03-24T21:09:06.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179012026-03-24T21:09:06.135ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
179022026-03-24T21:09:06.135ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
179032026-03-24T21:09:06.136ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 7d263e07-4fca-40c3-94c4-2ae7aef74d05 parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
179042026-03-24T21:09:06.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179052026-03-24T21:09:06.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179062026-03-24T21:09:06.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179072026-03-24T21:09:06.140ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
179082026-03-24T21:09:06.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179092026-03-24T21:09:06.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179102026-03-24T21:09:06.143ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
179112026-03-24T21:09:06.143ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
179122026-03-24T21:09:06.143ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 609.855205ms iteration = 31 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
179132026-03-24T21:09:06.240ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), time_created: 2026-03-24T20:57:02.694311Z, time_modified: 2026-03-24T20:57:02.694311Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
179142026-03-24T21:09:06.240ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
179152026-03-24T21:09:06.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179162026-03-24T21:09:06.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179172026-03-24T21:09:06.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179182026-03-24T21:09:06.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179192026-03-24T21:09:06.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179202026-03-24T21:09:06.249ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
179212026-03-24T21:09:06.249ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
179222026-03-24T21:09:06.250ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "2d24640d-2589-4164-b7c0-1939999dec08", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
179232026-03-24T21:09:06.250ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
179242026-03-24T21:09:06.250ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = Dns step id = Ensure
179252026-03-24T21:09:06.250ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
179262026-03-24T21:09:06.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179272026-03-24T21:09:06.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179282026-03-24T21:09:06.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179292026-03-24T21:09:06.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179302026-03-24T21:09:06.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179312026-03-24T21:09:06.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
179322026-03-24T21:09:06.298ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
179332026-03-24T21:09:06.298ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
179342026-03-24T21:09:06.298ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
179352026-03-24T21:09:06.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179362026-03-24T21:09:06.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179372026-03-24T21:09:06.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179382026-03-24T21:09:06.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
179392026-03-24T21:09:06.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179402026-03-24T21:09:06.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179412026-03-24T21:09:06.345ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
179422026-03-24T21:09:06.345ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
179432026-03-24T21:09:06.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179442026-03-24T21:09:06.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179452026-03-24T21:09:06.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179462026-03-24T21:09:06.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
179472026-03-24T21:09:06.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179482026-03-24T21:09:06.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179492026-03-24T21:09:06.352ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = e5047a31-94ba-4ade-ba09-3a3d4ed46ab1 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
179502026-03-24T21:09:06.352ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
179512026-03-24T21:09:06.352ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
179522026-03-24T21:09:06.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179532026-03-24T21:09:06.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179542026-03-24T21:09:06.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179552026-03-24T21:09:06.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179562026-03-24T21:09:06.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179572026-03-24T21:09:06.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
179582026-03-24T21:09:06.359ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
179592026-03-24T21:09:06.359ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
179602026-03-24T21:09:06.359ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
179612026-03-24T21:09:06.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179622026-03-24T21:09:06.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179632026-03-24T21:09:06.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179642026-03-24T21:09:06.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
179652026-03-24T21:09:06.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179662026-03-24T21:09:06.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179672026-03-24T21:09:06.366ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
179682026-03-24T21:09:06.366ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 0 version = 2
179692026-03-24T21:09:06.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179702026-03-24T21:09:06.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179712026-03-24T21:09:06.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179722026-03-24T21:09:06.371ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
179732026-03-24T21:09:06.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179742026-03-24T21:09:06.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179752026-03-24T21:09:06.373ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 2275d838-e328-4290-8b4e-3132b5f4b1b3 dns_zone_name = oxide.test found_so_far = 4 version = 2
179762026-03-24T21:09:06.373ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
179772026-03-24T21:09:06.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179782026-03-24T21:09:06.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179792026-03-24T21:09:06.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179802026-03-24T21:09:06.376ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
179812026-03-24T21:09:06.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179822026-03-24T21:09:06.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179832026-03-24T21:09:06.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179842026-03-24T21:09:06.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179852026-03-24T21:09:06.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179862026-03-24T21:09:06.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179872026-03-24T21:09:06.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179882026-03-24T21:09:06.383ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
179892026-03-24T21:09:06.385ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
179902026-03-24T21:09:06.385ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
179912026-03-24T21:09:06.385ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = OmicronZones step id = Cleanup
179922026-03-24T21:09:06.385ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = OmicronZones step id = Cleanup
179932026-03-24T21:09:06.385ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = PhysicalDisks step id = Cleanup
179942026-03-24T21:09:06.385ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = Clickhouse step id = Ensure
179952026-03-24T21:09:06.385ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = Clickhouse step id = Ensure
179962026-03-24T21:09:06.386ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
179972026-03-24T21:09:06.391ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "4568d393-7ec4-40c7-8fce-69c87df4dec3", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
179982026-03-24T21:09:06.391ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = SupportBundles step id = Cleanup
179992026-03-24T21:09:06.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180002026-03-24T21:09:06.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180012026-03-24T21:09:06.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180022026-03-24T21:09:06.394ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
180032026-03-24T21:09:06.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180042026-03-24T21:09:06.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180052026-03-24T21:09:06.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180062026-03-24T21:09:06.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180072026-03-24T21:09:06.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180082026-03-24T21:09:06.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
180092026-03-24T21:09:06.442ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = OmicronZones step id = Cleanup
180102026-03-24T21:09:06.443ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
180112026-03-24T21:09:06.443ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
180122026-03-24T21:09:06.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180132026-03-24T21:09:06.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180142026-03-24T21:09:06.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180152026-03-24T21:09:06.446ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
180162026-03-24T21:09:06.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180172026-03-24T21:09:06.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180182026-03-24T21:09:06.494ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
180192026-03-24T21:09:06.494ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9e1c09c1-3960-4ecf-a904-19e137699ab3 (blueprint), false)
180202026-03-24T21:09:06.494ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = Cockroach step id = Ensure
180212026-03-24T21:09:06.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180222026-03-24T21:09:06.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180232026-03-24T21:09:06.512ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
180242026-03-24T21:09:06.512ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 0df82ffc-d058-4f92-93d4-441dc4229abe step component = MgsUpdates step id = Ensure
180252026-03-24T21:09:06.512ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 16 reason = Dependency
180262026-03-24T21:09:06.512ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 38 reason = Dependency
180272026-03-24T21:09:06.512ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
180282026-03-24T21:09:06.512ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
180292026-03-24T21:09:06.512ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 990.079236ms iteration = 14 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":2981,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":96566103,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":96566063,"secs":0}},"event_index":1,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":96578834,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":238501511,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":238501461,"secs":0}},"event_index":2,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":335104726,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":76441451,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":76441421,"secs":0}},"event_index":3,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":411575508,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5851397,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5851348,"secs":0}},"event_index":4,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":417440197,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":310210664,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":310210624,"secs":0}},"event_index":5,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":727660911,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":135287937,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":135287887,"secs":0}},"event_index":6,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":862955999,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5410,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5380,"secs":0}},"event_index":7,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":862972839,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2270,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2230,"secs":0}},"event_index":8,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":862977019,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3989,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3949,"secs":0}},"event_index":9,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":862982799,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1590,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1560,"secs":0}},"event_index":10,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":862985730,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6016062,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":6016022,"secs":0}},"event_index":11,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":869002852,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":51542220,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":51542170,"secs":0}},"event_index":12,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":920550872,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":51362865,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":51362835,"secs":0}},"event_index":13,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":971929007,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":17838441,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":17838412,"secs":0}},"event_index":14,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":989775738,"secs":0}},{"data":{"attempt_elapsed":{"nanos":4530,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4490,"secs":0}},"event_index":15,"execution_id":"524562e8-f6f1-4356-acee-16508ef5b440","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":989786639,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3"}
180302026-03-24T21:09:06.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
180312026-03-24T21:09:06.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
180322026-03-24T21:09:06.514ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180332026-03-24T21:09:06.514ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180342026-03-24T21:09:06.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
180352026-03-24T21:09:06.516ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
180362026-03-24T21:09:06.516ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
180372026-03-24T21:09:06.516ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
180382026-03-24T21:09:06.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
180392026-03-24T21:09:06.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
180402026-03-24T21:09:06.518ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
180412026-03-24T21:09:06.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
180422026-03-24T21:09:06.520ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180432026-03-24T21:09:06.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
180442026-03-24T21:09:06.521ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
180452026-03-24T21:09:06.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
180462026-03-24T21:09:06.523ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
180472026-03-24T21:09:06.523ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 11.463149ms iteration = 38 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
180482026-03-24T21:09:06.558ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(da898c00-a280-40a9-aa3c-5dfd1b54176d, 0), (77c2ae7d-26c8-4055-a6f9-3aab675a3933, 0), (9b8824f4-eaf4-42c4-80a6-3c8210ba299f, 0), (fae51d14-063d-4c09-aff4-5b856f69c82a, 0)] collection_id = da898c00-a280-40a9-aa3c-5dfd1b54176d
180492026-03-24T21:09:06.558ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
180502026-03-24T21:09:06.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
180512026-03-24T21:09:06.682ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = da898c00-a280-40a9-aa3c-5dfd1b54176d file = nexus/db-queries/src/db/datastore/inventory.rs:2445 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 102 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 84 nlast_reconciliation_disk_results = 9 nlast_reconciliation_measurements = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmeasurement_manifest_non_boot = 0 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 84 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_measurements = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
180522026-03-24T21:09:06.682ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
180532026-03-24T21:09:06.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
180542026-03-24T21:09:06.687ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(77c2ae7d-26c8-4055-a6f9-3aab675a3933, 0), (9b8824f4-eaf4-42c4-80a6-3c8210ba299f, 0), (fae51d14-063d-4c09-aff4-5b856f69c82a, 0)]
180552026-03-24T21:09:06.687ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 900643981 } })
180562026-03-24T21:09:06.689ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1345db7a-abc1-4a4c-a49c-e6c56102e521.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1920db37-87f5-4a96-b3ef-1db2eb59e9a5.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6b02d2eb-4373-4f12-8efc-4c0e3f4fa77b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("709c4704-f768-4640-a198-a23621ed6e11.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("c3b068bb-6f9c-4c4e-b559-e3154ac75f47.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 1532, tv_nsec: 902257044 } })
180572026-03-24T21:09:06.689ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
180582026-03-24T21:09:06.689ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
180592026-03-24T21:09:06.689ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
180602026-03-24T21:09:06.690ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ec9f7e9-4eb1-4c62-ac40-5c47535d8f7c", "content-length": "476", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180612026-03-24T21:09:06.690ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
180622026-03-24T21:09:06.690ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8453e83f-7a87-428e-b940-e96eaf32e865", "content-length": "734", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180632026-03-24T21:09:06.690ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
180642026-03-24T21:09:06.691ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d8b7083-51de-4357-a59d-bceb677329e1", "content-length": "111", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180652026-03-24T21:09:06.691ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
180662026-03-24T21:09:06.691ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fe2a79c-4f90-48b6-aeac-310325748bf1", "content-length": "111", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180672026-03-24T21:09:06.691ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
180682026-03-24T21:09:06.691ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "36adc5cf-64f7-4be2-9617-c185d3f3c1c2", "content-length": "163", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180692026-03-24T21:09:06.691ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
180702026-03-24T21:09:06.692ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd583b03-0064-422c-ad10-6afb274901f3", "content-length": "163", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180712026-03-24T21:09:06.692ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
180722026-03-24T21:09:06.692ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "13bbe8c5-47d8-496c-95fe-1ffe1ea4d45a", "content-length": "166", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180732026-03-24T21:09:06.692ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
180742026-03-24T21:09:06.693ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68b9fdd7-572d-4bfd-8eda-a2bede77e018", "content-length": "166", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180752026-03-24T21:09:06.693ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
180762026-03-24T21:09:06.693ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4532852-22d9-4ee4-bcfe-0350364484a0", "content-length": "702", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180772026-03-24T21:09:06.693ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
180782026-03-24T21:09:06.693ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "537bc1cb-456b-4aa1-b5ca-7c4385e9719e", "content-length": "727", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180792026-03-24T21:09:06.693ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
180802026-03-24T21:09:06.693ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "40f6fcbc-1beb-465b-8edf-512691a5e7b1", "content-length": "729", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180812026-03-24T21:09:06.693ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
180822026-03-24T21:09:06.694ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "23bceb43-c24e-4e08-8690-5fa4142fd5c1", "content-length": "728", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180832026-03-24T21:09:06.694ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
180842026-03-24T21:09:06.694ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d972a87b-d724-48d3-8ae7-a37e8804c84d", "content-length": "722", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180852026-03-24T21:09:06.694ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
180862026-03-24T21:09:06.694ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "526b022a-ab06-459a-ba0a-f01425fb417e", "content-length": "10", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180872026-03-24T21:09:06.694ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
180882026-03-24T21:09:06.694ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "97df1d0e-22d9-424b-a6a9-a80315577e5c", "content-length": "144", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180892026-03-24T21:09:06.694ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
180902026-03-24T21:09:06.695ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b1e0c6c-d8f2-46c4-8546-1274cc0c247c", "content-length": "147", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180912026-03-24T21:09:06.695ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
180922026-03-24T21:09:06.695ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0da3b8bf-5f4c-49b7-929d-af317a54d249", "content-length": "109", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180932026-03-24T21:09:06.695ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
180942026-03-24T21:09:06.695ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d7de781-17a7-4cbd-9582-3eca2f47705c", "content-length": "109", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180952026-03-24T21:09:06.695ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
180962026-03-24T21:09:06.695ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a2471d7-587d-4c6f-9609-4a5b8cd88a09", "content-length": "163", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180972026-03-24T21:09:06.695ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
180982026-03-24T21:09:06.696ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "00fc3fc4-f1d7-4345-9fda-7cfa16b0a52e", "content-length": "163", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
180992026-03-24T21:09:06.696ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
181002026-03-24T21:09:06.696ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b86b234-85ed-4296-b764-4fd01dd8375c", "content-length": "166", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181012026-03-24T21:09:06.696ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
181022026-03-24T21:09:06.697ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9697c57-48ba-462e-97ae-4075b8708be2", "content-length": "166", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181032026-03-24T21:09:06.697ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
181042026-03-24T21:09:06.697ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b402595b-de94-491e-9d81-919b87be7a78", "content-length": "702", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181052026-03-24T21:09:06.697ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
181062026-03-24T21:09:06.697ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a81af68-7977-4d1b-8e58-f8d3bae38281", "content-length": "727", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181072026-03-24T21:09:06.697ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
181082026-03-24T21:09:06.697ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e759f9d3-86b9-4485-bb22-a5319dd692d9", "content-length": "729", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181092026-03-24T21:09:06.697ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
181102026-03-24T21:09:06.698ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "46373624-31b8-415d-9f93-fd07208a4d19", "content-length": "728", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181112026-03-24T21:09:06.698ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
181122026-03-24T21:09:06.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
181132026-03-24T21:09:06.700ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181142026-03-24T21:09:06.702ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
181152026-03-24T21:09:06.702ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
181162026-03-24T21:09:06.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
181172026-03-24T21:09:06.704ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181182026-03-24T21:09:06.706ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
181192026-03-24T21:09:06.706ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
181202026-03-24T21:09:06.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
181212026-03-24T21:09:06.709ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
181222026-03-24T21:09:06.709ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
181232026-03-24T21:09:06.786ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fa34317-7cf3-41ef-84e7-0cdb7b365de9", "content-length": "110902", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181242026-03-24T21:09:06.786ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
181252026-03-24T21:09:06.786ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
181262026-03-24T21:09:06.786ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
181272026-03-24T21:09:06.786ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
181282026-03-24T21:09:06.786ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
181292026-03-24T21:09:06.786ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
181302026-03-24T21:09:06.786ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
181312026-03-24T21:09:06.786ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
181322026-03-24T21:09:06.787ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "94e80e37-c7fc-44a7-bbfc-781c866864a3", "content-length": "64", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181332026-03-24T21:09:06.787ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
181342026-03-24T21:09:06.787ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "99e1d9ae-a159-43fd-8fd1-d22e9a542bd2", "content-length": "64", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181352026-03-24T21:09:06.787ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
181362026-03-24T21:09:06.787ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7d9d057-280d-41f0-b385-69bdfd90e492", "content-length": "64", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181372026-03-24T21:09:06.787ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
181382026-03-24T21:09:06.787ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "02fad6a2-a6b2-4d19-a647-89da8c880a15", "content-length": "64", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181392026-03-24T21:09:06.787ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1335d2b-dfa7-4a95-b38c-f1827a6180ee", "content-length": "64", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181402026-03-24T21:09:06.787ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
181412026-03-24T21:09:06.787ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
181422026-03-24T21:09:06.794ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a573c25c-38cd-4ac8-82b5-12c2628f5f1a", "content-length": "407408", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181432026-03-24T21:09:06.794ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "927c5a0d-0d19-478e-a91f-51e2e56c618a", "content-length": "419386", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181442026-03-24T21:09:06.795ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c17af9e0-5a86-4e65-83e8-2c8fa35be49c", "content-length": "438359", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181452026-03-24T21:09:06.797ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ef7646b-2c55-46f2-91fb-60bf80356536", "content-length": "411111", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181462026-03-24T21:09:06.797ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "592f30c6-a63f-4032-a298-1b098409a624", "content-length": "411594", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181472026-03-24T21:09:06.800ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
181482026-03-24T21:09:06.800ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
181492026-03-24T21:09:06.800ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
181502026-03-24T21:09:06.800ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
181512026-03-24T21:09:06.800ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
181522026-03-24T21:09:06.801ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = ea49db54-0bfa-4e3d-bc93-404c287645e6 (omicron_zone)
181532026-03-24T21:09:06.801ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
181542026-03-24T21:09:06.804ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "79deccc0-47ba-4193-a4cd-d8d5b003ecc0", "content-length": "128", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181552026-03-24T21:09:06.804ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
181562026-03-24T21:09:06.806ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
181572026-03-24T21:09:06.806ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
181582026-03-24T21:09:06.807ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0818ac13-4035-4213-825a-24c16eba9a0a", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181592026-03-24T21:09:06.807ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 27f3a9c5-67c7-41c5-994d-9db513cda557 (omicron_zone)
181602026-03-24T21:09:06.807ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
181612026-03-24T21:09:06.807ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
181622026-03-24T21:09:06.807ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e65ee517-ff16-4c93-b8c7-f002645947bb", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181632026-03-24T21:09:06.808ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 2f7ea2ed-fd45-4cd7-9fb1-f48119541118 (omicron_zone)
181642026-03-24T21:09:06.808ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
181652026-03-24T21:09:06.808ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
181662026-03-24T21:09:06.808ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5aff9086-9208-438a-a592-b9f1d88b1c11", "content-length": "9552", "date": "Tue, 24 Mar 2026 21:09:06 GMT"} })
181672026-03-24T21:09:06.808ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 7f3bc00f-0320-479a-a8fb-f67e80ed5929 (omicron_zone)
181682026-03-24T21:09:06.808ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
181692026-03-24T21:09:06.808ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
181702026-03-24T21:09:06.808ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
181712026-03-24T21:09:06.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
181722026-03-24T21:09:06.811ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181732026-03-24T21:09:06.814ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
181742026-03-24T21:09:06.814ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
181752026-03-24T21:09:06.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
181762026-03-24T21:09:07.048ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 436b829b-e7b8-47ef-be41-4cbb6dd3bf73 file = nexus/db-queries/src/db/datastore/inventory.rs:1757
181772026-03-24T21:09:07.048ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 436b829b-e7b8-47ef-be41-4cbb6dd3bf73 time_started = 2026-03-24 21:09:06.689867 UTC
181782026-03-24T21:09:07.048ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 65 reason = Dependency
181792026-03-24T21:09:07.048ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
181802026-03-24T21:09:07.048ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 536.556128ms iteration = 16 status = {"collection_id":"436b829b-e7b8-47ef-be41-4cbb6dd3bf73","time_done":"2026-03-24 21:09:06.808624 UTC","time_started":"2026-03-24 21:09:06.689867 UTC"}
181812026-03-24T21:09:07.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
181822026-03-24T21:09:07.050ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181832026-03-24T21:09:07.054ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
181842026-03-24T21:09:07.054ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
181852026-03-24T21:09:07.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
181862026-03-24T21:09:07.145ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
181872026-03-24T21:09:07.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
181882026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 30 reason = Dependency
181892026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 143.728185ms iteration = 65 status = {"Loaded":{"collection_id":"55464b9a-4fc2-45bb-97a5-05ea8d50a065","time_loaded":"2026-03-24T21:09:07.048782154Z","time_started":"2026-03-24T21:09:06.749597Z"}}
181902026-03-24T21:09:07.192ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:453
181912026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 32 reason = Dependency
181922026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 42 reason = Dependency
181932026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 14.94\u{b5}s iteration = 30 status = {"disabled":true,"empty_groups_marked":0,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
181942026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 20 reason = Dependency
181952026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
181962026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
181972026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
181982026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
181992026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
182002026-03-24T21:09:07.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182012026-03-24T21:09:07.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182022026-03-24T21:09:07.194ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182032026-03-24T21:09:07.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182042026-03-24T21:09:07.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
182052026-03-24T21:09:07.197ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
182062026-03-24T21:09:07.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
182072026-03-24T21:09:07.197ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
182082026-03-24T21:09:07.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
182092026-03-24T21:09:07.197ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
182102026-03-24T21:09:07.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182112026-03-24T21:09:07.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
182122026-03-24T21:09:07.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182132026-03-24T21:09:07.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182142026-03-24T21:09:07.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182152026-03-24T21:09:07.199ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
182162026-03-24T21:09:07.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
182172026-03-24T21:09:07.201ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182182026-03-24T21:09:07.201ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
182192026-03-24T21:09:07.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182202026-03-24T21:09:07.202ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
182212026-03-24T21:09:07.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
182222026-03-24T21:09:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182232026-03-24T21:09:07.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
182242026-03-24T21:09:07.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
182252026-03-24T21:09:07.203ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
182262026-03-24T21:09:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
182272026-03-24T21:09:07.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182282026-03-24T21:09:07.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
182292026-03-24T21:09:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182302026-03-24T21:09:07.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
182312026-03-24T21:09:07.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182322026-03-24T21:09:07.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182332026-03-24T21:09:07.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
182342026-03-24T21:09:07.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
182352026-03-24T21:09:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
182362026-03-24T21:09:07.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
182372026-03-24T21:09:07.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182382026-03-24T21:09:07.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182392026-03-24T21:09:07.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182402026-03-24T21:09:07.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182412026-03-24T21:09:07.212ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
182422026-03-24T21:09:07.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
182432026-03-24T21:09:07.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
182442026-03-24T21:09:07.213ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
182452026-03-24T21:09:07.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182462026-03-24T21:09:07.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182472026-03-24T21:09:07.214ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
182482026-03-24T21:09:07.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
182492026-03-24T21:09:07.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
182502026-03-24T21:09:07.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182512026-03-24T21:09:07.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182522026-03-24T21:09:07.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182532026-03-24T21:09:07.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182542026-03-24T21:09:07.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182552026-03-24T21:09:07.218ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
182562026-03-24T21:09:07.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
182572026-03-24T21:09:07.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
182582026-03-24T21:09:07.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182592026-03-24T21:09:07.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182602026-03-24T21:09:07.222ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
182612026-03-24T21:09:07.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
182622026-03-24T21:09:07.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
182632026-03-24T21:09:07.224ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
182642026-03-24T21:09:07.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
182652026-03-24T21:09:07.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
182662026-03-24T21:09:07.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
182672026-03-24T21:09:07.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182682026-03-24T21:09:07.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182692026-03-24T21:09:07.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182702026-03-24T21:09:07.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182712026-03-24T21:09:07.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182722026-03-24T21:09:07.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182732026-03-24T21:09:07.233ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
182742026-03-24T21:09:07.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
182752026-03-24T21:09:07.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
182762026-03-24T21:09:07.233ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
182772026-03-24T21:09:07.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182782026-03-24T21:09:07.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182792026-03-24T21:09:07.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182802026-03-24T21:09:07.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182812026-03-24T21:09:07.238ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
182822026-03-24T21:09:07.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
182832026-03-24T21:09:07.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
182842026-03-24T21:09:07.239ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
182852026-03-24T21:09:07.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182862026-03-24T21:09:07.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182872026-03-24T21:09:07.241ZINFOnexus (BackgroundTasks): all LocalStorageUnencrypted datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_unencrypted_dataset.rs:149 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
182882026-03-24T21:09:07.241ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 48.95759ms iteration = 20 status = {"blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","inventory_collection_id":"55464b9a-4fc2-45bb-97a5-05ea8d50a065","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_unencrypted_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
182892026-03-24T21:09:07.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182902026-03-24T21:09:07.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182912026-03-24T21:09:07.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182922026-03-24T21:09:07.247ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ea7982bc-a3b0-44a4-a88f-708033107b2c, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
182932026-03-24T21:09:07.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182942026-03-24T21:09:07.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182952026-03-24T21:09:07.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
182962026-03-24T21:09:07.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
182972026-03-24T21:09:07.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182982026-03-24T21:09:07.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
182992026-03-24T21:09:07.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183002026-03-24T21:09:07.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183012026-03-24T21:09:07.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183022026-03-24T21:09:07.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183032026-03-24T21:09:07.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183042026-03-24T21:09:07.260ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
183052026-03-24T21:09:07.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183062026-03-24T21:09:07.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183072026-03-24T21:09:07.263ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 71.151288ms iteration = 42 status = {"physical_disks_added":0}
183082026-03-24T21:09:07.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183092026-03-24T21:09:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183102026-03-24T21:09:07.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183112026-03-24T21:09:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183122026-03-24T21:09:07.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183132026-03-24T21:09:07.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
183142026-03-24T21:09:07.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183152026-03-24T21:09:07.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183162026-03-24T21:09:07.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183172026-03-24T21:09:07.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
183182026-03-24T21:09:07.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183192026-03-24T21:09:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183202026-03-24T21:09:07.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183212026-03-24T21:09:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183222026-03-24T21:09:07.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183232026-03-24T21:09:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183242026-03-24T21:09:07.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183252026-03-24T21:09:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
183262026-03-24T21:09:07.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183272026-03-24T21:09:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183282026-03-24T21:09:07.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183292026-03-24T21:09:07.302ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: b1a060c9-f0ec-4cb6-8473-b3d194aefa8a, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
183302026-03-24T21:09:07.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183312026-03-24T21:09:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183322026-03-24T21:09:07.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183332026-03-24T21:09:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183342026-03-24T21:09:07.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183352026-03-24T21:09:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183362026-03-24T21:09:07.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183372026-03-24T21:09:07.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
183382026-03-24T21:09:07.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183392026-03-24T21:09:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183402026-03-24T21:09:07.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183412026-03-24T21:09:07.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183422026-03-24T21:09:07.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183432026-03-24T21:09:07.315ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
183442026-03-24T21:09:07.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183452026-03-24T21:09:07.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183462026-03-24T21:09:07.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183472026-03-24T21:09:07.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183482026-03-24T21:09:07.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183492026-03-24T21:09:07.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
183502026-03-24T21:09:07.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183512026-03-24T21:09:07.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183522026-03-24T21:09:07.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183532026-03-24T21:09:07.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183542026-03-24T21:09:07.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183552026-03-24T21:09:07.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
183562026-03-24T21:09:07.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183572026-03-24T21:09:07.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183582026-03-24T21:09:07.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183592026-03-24T21:09:07.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183602026-03-24T21:09:07.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183612026-03-24T21:09:07.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
183622026-03-24T21:09:07.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183632026-03-24T21:09:07.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183642026-03-24T21:09:07.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183652026-03-24T21:09:07.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183662026-03-24T21:09:07.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183672026-03-24T21:09:07.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
183682026-03-24T21:09:07.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183692026-03-24T21:09:07.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183702026-03-24T21:09:07.383ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
183712026-03-24T21:09:07.383ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:243 reason = no target release is currently set
183722026-03-24T21:09:07.383ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = bfdf90b9-5f9d-4f92-9e38-53287fc4206b parent_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 sled_id = b4293d4f-3308-4dfa-b261-a103215913e4
183732026-03-24T21:09:07.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183742026-03-24T21:09:07.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183752026-03-24T21:09:07.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183762026-03-24T21:09:07.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
183772026-03-24T21:09:07.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
183782026-03-24T21:09:07.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
183792026-03-24T21:09:07.391ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
183802026-03-24T21:09:07.391ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3
183812026-03-24T21:09:07.391ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 198.960765ms iteration = 32 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4: 28 zones\\n","sleds have measurements with image sources not set to Artifact:\\n- sled b4293d4f-3308-4dfa-b261-a103215913e4 set to install dataset\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"measurement_updates":{"type":"blocked_add_update"},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_measurements":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_measurements_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
183822026-03-24T21:09:08.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 364 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 2724c687-5e0c-42db-8c99-483ba8662a45 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
183832026-03-24T21:09:10.534ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 146 reason = Timeout
183842026-03-24T21:09:10.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
183852026-03-24T21:09:10.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
183862026-03-24T21:09:10.537ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183872026-03-24T21:09:10.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
183882026-03-24T21:09:10.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
183892026-03-24T21:09:10.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
183902026-03-24T21:09:10.542ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
183912026-03-24T21:09:10.542ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.043567ms iteration = 146 status = {"config_updated":false}
183922026-03-24T21:09:15.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
183932026-03-24T21:09:15.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 147 reason = Timeout
183942026-03-24T21:09:15.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
183952026-03-24T21:09:15.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
183962026-03-24T21:09:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
183972026-03-24T21:09:15.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
183982026-03-24T21:09:15.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183992026-03-24T21:09:15.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184002026-03-24T21:09:15.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
184012026-03-24T21:09:15.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
184022026-03-24T21:09:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
184032026-03-24T21:09:15.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
184042026-03-24T21:09:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
184052026-03-24T21:09:15.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
184062026-03-24T21:09:15.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
184072026-03-24T21:09:15.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
184082026-03-24T21:09:15.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
184092026-03-24T21:09:15.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.952935ms iteration = 147 status = {"config_updated":false}
184102026-03-24T21:09:15.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184112026-03-24T21:09:15.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
184122026-03-24T21:09:15.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
184132026-03-24T21:09:15.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
184142026-03-24T21:09:15.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
184152026-03-24T21:09:15.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
184162026-03-24T21:09:15.562ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
184172026-03-24T21:09:15.562ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.734885ms iteration = 74 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
184182026-03-24T21:09:15.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
184192026-03-24T21:09:15.601ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
184202026-03-24T21:09:15.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
184212026-03-24T21:09:15.603ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.650523ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
184222026-03-24T21:09:18.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 366 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 9ad42623-3243-4ac7-a94f-c7726a6f185e response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
184232026-03-24T21:09:20.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 66 reason = Timeout
184242026-03-24T21:09:20.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 148 reason = Timeout
184252026-03-24T21:09:20.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
184262026-03-24T21:09:20.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
184272026-03-24T21:09:20.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
184282026-03-24T21:09:20.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
184292026-03-24T21:09:20.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184302026-03-24T21:09:20.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184312026-03-24T21:09:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
184322026-03-24T21:09:20.538ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
184332026-03-24T21:09:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
184342026-03-24T21:09:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
184352026-03-24T21:09:20.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
184362026-03-24T21:09:20.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
184372026-03-24T21:09:20.540ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 55464b9a-4fc2-45bb-97a5-05ea8d50a065 original_time_started = 2026-03-24 21:09:06.749597 UTC
184382026-03-24T21:09:20.540ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.227015ms iteration = 66 status = {"Loaded":{"collection_id":"55464b9a-4fc2-45bb-97a5-05ea8d50a065","time_loaded":"2026-03-24T21:09:20.533508761Z","time_started":"2026-03-24T21:09:06.749597Z"}}
184392026-03-24T21:09:20.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
184402026-03-24T21:09:20.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.561874ms iteration = 148 status = {"config_updated":false}
184412026-03-24T21:09:20.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 50 reason = Timeout
184422026-03-24T21:09:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
184432026-03-24T21:09:20.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 50 reason = Timeout
184442026-03-24T21:09:20.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
184452026-03-24T21:09:20.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
184462026-03-24T21:09:20.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
184472026-03-24T21:09:20.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
184482026-03-24T21:09:20.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
184492026-03-24T21:09:20.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184502026-03-24T21:09:20.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184512026-03-24T21:09:20.586ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
184522026-03-24T21:09:20.586ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
184532026-03-24T21:09:20.586ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
184542026-03-24T21:09:20.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
184552026-03-24T21:09:20.588ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
184562026-03-24T21:09:20.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.666983ms iteration = 50 status = "NoSitrep"
184572026-03-24T21:09:20.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
184582026-03-24T21:09:20.588ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
184592026-03-24T21:09:20.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
184602026-03-24T21:09:20.590ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184612026-03-24T21:09:20.591ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
184622026-03-24T21:09:20.596ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
184632026-03-24T21:09:20.596ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
184642026-03-24T21:09:20.596ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.509404ms iteration = 50 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
184652026-03-24T21:09:25.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
184662026-03-24T21:09:25.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 149 reason = Timeout
184672026-03-24T21:09:25.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
184682026-03-24T21:09:25.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
184692026-03-24T21:09:25.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
184702026-03-24T21:09:25.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
184712026-03-24T21:09:25.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184722026-03-24T21:09:25.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184732026-03-24T21:09:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
184742026-03-24T21:09:25.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
184752026-03-24T21:09:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
184762026-03-24T21:09:25.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
184772026-03-24T21:09:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
184782026-03-24T21:09:25.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
184792026-03-24T21:09:25.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
184802026-03-24T21:09:25.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.220395ms iteration = 149 status = {"config_updated":false}
184812026-03-24T21:09:25.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
184822026-03-24T21:09:25.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
184832026-03-24T21:09:25.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184842026-03-24T21:09:25.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
184852026-03-24T21:09:25.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
184862026-03-24T21:09:25.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
184872026-03-24T21:09:25.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
184882026-03-24T21:09:25.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
184892026-03-24T21:09:25.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
184902026-03-24T21:09:25.564ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
184912026-03-24T21:09:25.564ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.683177ms iteration = 75 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
184922026-03-24T21:09:25.603ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
184932026-03-24T21:09:25.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
184942026-03-24T21:09:25.604ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 43.38557ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
184952026-03-24T21:09:28.558ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 362 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 24f9205b-4e7f-4233-bb14-5a8baf16120b response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
184962026-03-24T21:09:30.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 150 reason = Timeout
184972026-03-24T21:09:30.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
184982026-03-24T21:09:30.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
184992026-03-24T21:09:30.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185002026-03-24T21:09:30.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
185012026-03-24T21:09:30.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
185022026-03-24T21:09:30.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
185032026-03-24T21:09:30.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
185042026-03-24T21:09:30.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.802824ms iteration = 150 status = {"config_updated":false}
185052026-03-24T21:09:35.521ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
185062026-03-24T21:09:35.521ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 26 reason = Timeout
185072026-03-24T21:09:35.521ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
185082026-03-24T21:09:35.521ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
185092026-03-24T21:09:35.521ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
185102026-03-24T21:09:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
185112026-03-24T21:09:35.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
185122026-03-24T21:09:35.525ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 738140926 } })
185132026-03-24T21:09:35.525ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 738140976 } })
185142026-03-24T21:09:35.527ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1392
185152026-03-24T21:09:35.527ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 740569152 } })
185162026-03-24T21:09:35.527ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
185172026-03-24T21:09:35.527ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 6.253439ms iteration = 26 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
185182026-03-24T21:09:35.528ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1419 zone_address = fd00:1122:3344:101::2
185192026-03-24T21:09:35.528ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
185202026-03-24T21:09:35.529ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d07fa8c-ca24-4be9-9c4b-0900984452fc", "content-length": "26", "date": "Tue, 24 Mar 2026 21:09:35 GMT"} })
185212026-03-24T21:09:35.529ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1422 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
185222026-03-24T21:09:35.529ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n}
185232026-03-24T21:09:35.529ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 742666268 } })
185242026-03-24T21:09:35.531ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 9.911297ms iteration = 26 status = {}
185252026-03-24T21:09:35.531ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.212175ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
185262026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
185272026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
185282026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
185292026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
185302026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
185312026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
185322026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 43 reason = Timeout
185332026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 67 reason = Timeout
185342026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
185352026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
185362026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 151 reason = Timeout
185372026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 39 reason = Timeout
185382026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
185392026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
185402026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
185412026-03-24T21:09:35.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
185422026-03-24T21:09:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
185432026-03-24T21:09:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
185442026-03-24T21:09:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
185452026-03-24T21:09:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
185462026-03-24T21:09:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
185472026-03-24T21:09:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
185482026-03-24T21:09:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
185492026-03-24T21:09:35.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
185502026-03-24T21:09:35.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185512026-03-24T21:09:35.536ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185522026-03-24T21:09:35.536ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185532026-03-24T21:09:35.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185542026-03-24T21:09:35.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185552026-03-24T21:09:35.537ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
185562026-03-24T21:09:35.537ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185572026-03-24T21:09:35.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
185582026-03-24T21:09:35.539ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185592026-03-24T21:09:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
185602026-03-24T21:09:35.540ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
185612026-03-24T21:09:35.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
185622026-03-24T21:09:35.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
185632026-03-24T21:09:35.541ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
185642026-03-24T21:09:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
185652026-03-24T21:09:35.542ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
185662026-03-24T21:09:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
185672026-03-24T21:09:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
185682026-03-24T21:09:35.542ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
185692026-03-24T21:09:35.542ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
185702026-03-24T21:09:35.542ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
185712026-03-24T21:09:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
185722026-03-24T21:09:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
185732026-03-24T21:09:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
185742026-03-24T21:09:35.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
185752026-03-24T21:09:35.543ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
185762026-03-24T21:09:35.543ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
185772026-03-24T21:09:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
185782026-03-24T21:09:35.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
185792026-03-24T21:09:35.544ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185802026-03-24T21:09:35.545ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
185812026-03-24T21:09:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b4293d4f-3308-4dfa-b261-a103215913e4 (sled), lookup_type: ById(b4293d4f-3308-4dfa-b261-a103215913e4) } result = Ok(())
185822026-03-24T21:09:35.545ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 55464b9a-4fc2-45bb-97a5-05ea8d50a065 original_time_started = 2026-03-24 21:09:06.749597 UTC
185832026-03-24T21:09:35.545ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
185842026-03-24T21:09:35.545ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 11.622694ms iteration = 67 status = {"Loaded":{"collection_id":"55464b9a-4fc2-45bb-97a5-05ea8d50a065","time_loaded":"2026-03-24T21:09:35.533738659Z","time_started":"2026-03-24T21:09:06.749597Z"}}
185852026-03-24T21:09:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
185862026-03-24T21:09:35.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
185872026-03-24T21:09:35.545ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 758700981 } })
185882026-03-24T21:09:35.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
185892026-03-24T21:09:35.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
185902026-03-24T21:09:35.546ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1392 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
185912026-03-24T21:09:35.546ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 759470671 } })
185922026-03-24T21:09:35.546ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
185932026-03-24T21:09:35.546ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 12.931039ms iteration = 151 status = {"config_updated":false}
185942026-03-24T21:09:35.547ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
185952026-03-24T21:09:35.547ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
185962026-03-24T21:09:35.547ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185972026-03-24T21:09:35.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185982026-03-24T21:09:35.547ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1419 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d zone_address = fd00:1122:3344:101::2
185992026-03-24T21:09:35.547ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
186002026-03-24T21:09:35.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
186012026-03-24T21:09:35.547ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 14.168852ms iteration = 26 status = 0
186022026-03-24T21:09:35.548ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
186032026-03-24T21:09:35.548ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b7ba1ac-d450-4bfd-9edd-9e9d2b1594e5", "content-length": "26", "date": "Tue, 24 Mar 2026 21:09:35 GMT"} })
186042026-03-24T21:09:35.548ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1422 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
186052026-03-24T21:09:35.548ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1457 mappings = {\n switch0: fd00:1122:3344:101::2,\n} rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
186062026-03-24T21:09:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
186072026-03-24T21:09:35.548ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
186082026-03-24T21:09:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
186092026-03-24T21:09:35.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
186102026-03-24T21:09:35.550ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 763199242 } })
186112026-03-24T21:09:35.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
186122026-03-24T21:09:35.550ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
186132026-03-24T21:09:35.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
186142026-03-24T21:09:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
186152026-03-24T21:09:35.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
186162026-03-24T21:09:35.552ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b4293d4f-3308-4dfa-b261-a103215913e4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1561, tv_nsec: 765779452 } })
186172026-03-24T21:09:35.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
186182026-03-24T21:09:35.554ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
186192026-03-24T21:09:35.554ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
186202026-03-24T21:09:35.557ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:195 port = SwitchPort { id: b410756b-fa8a-4c74-8526-ed2ff6385bd6, rack_id: ee0b7d07-281d-4353-9052-3357eca0506d, port_name: Name(Name("qsfp0")), port_settings_id: Some(652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3), switch_slot: Switch0 } rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
186212026-03-24T21:09:35.557ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
186222026-03-24T21:09:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
186232026-03-24T21:09:35.558ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
186242026-03-24T21:09:35.558ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
186252026-03-24T21:09:35.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
186262026-03-24T21:09:35.560ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
186272026-03-24T21:09:35.560ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
186282026-03-24T21:09:35.560ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
186292026-03-24T21:09:35.560ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
186302026-03-24T21:09:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
186312026-03-24T21:09:35.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
186322026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
186332026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
186342026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 39 reason = Timeout
186352026-03-24T21:09:35.562ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
186362026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
186372026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
186382026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
186392026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
186402026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
186412026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
186422026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
186432026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
186442026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
186452026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
186462026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
186472026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
186482026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
186492026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
186502026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
186512026-03-24T21:09:35.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
186522026-03-24T21:09:35.564ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186532026-03-24T21:09:35.565ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186542026-03-24T21:09:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
186552026-03-24T21:09:35.569ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
186562026-03-24T21:09:35.569ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
186572026-03-24T21:09:35.569ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
186582026-03-24T21:09:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
186592026-03-24T21:09:35.569ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
186602026-03-24T21:09:35.569ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
186612026-03-24T21:09:35.569ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
186622026-03-24T21:09:35.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
186632026-03-24T21:09:35.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
186642026-03-24T21:09:35.572ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186652026-03-24T21:09:35.574ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 40.696458ms iteration = 43 status = {"physical_disks_added":0}
186662026-03-24T21:09:35.575ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
186672026-03-24T21:09:35.575ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
186682026-03-24T21:09:35.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
186692026-03-24T21:09:35.582ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 26 reason = Timeout
186702026-03-24T21:09:35.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 51 reason = Timeout
186712026-03-24T21:09:35.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 51 reason = Timeout
186722026-03-24T21:09:35.582ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 19.581\u{b5}s iteration = 26 status = {"disabled":true,"errors":[],"sps":[]}
186732026-03-24T21:09:35.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
186742026-03-24T21:09:35.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
186752026-03-24T21:09:35.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
186762026-03-24T21:09:35.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
186772026-03-24T21:09:35.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
186782026-03-24T21:09:35.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
186792026-03-24T21:09:35.584ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2110 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
186802026-03-24T21:09:35.584ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2137 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_location = switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 652f65d8-fb9a-47b4-b8c9-e1146fcc6ba3
186812026-03-24T21:09:35.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186822026-03-24T21:09:35.584ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:399 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
186832026-03-24T21:09:35.584ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:403 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: 1 })}}
186842026-03-24T21:09:35.584ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
186852026-03-24T21:09:35.584ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d routes = {}
186862026-03-24T21:09:35.585ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:438 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d
186872026-03-24T21:09:35.585ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
186882026-03-24T21:09:35.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
186892026-03-24T21:09:35.587ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
186902026-03-24T21:09:35.587ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
186912026-03-24T21:09:35.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
186922026-03-24T21:09:35.593ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: Static { ip_net: UplinkIpNet(V4(Ipv4Net { addr: 10.151.1.174, width: 24 })) }, vlan_id: None }], lldp: Some(LldpPortConfig { status: Disabled, chassis_id: None, port_id: None, port_description: None, system_name: None, system_description: None, management_addrs: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:487 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch_slot = switch0
186932026-03-24T21:09:35.594ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = ee0b7d07-281d-4353-9052-3357eca0506d uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
186942026-03-24T21:09:35.594ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 11.993524ms iteration = 51 status = "NoSitrep"
186952026-03-24T21:09:35.595ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186962026-03-24T21:09:35.597ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
186972026-03-24T21:09:35.597ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 63.81044ms iteration = 76 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
186982026-03-24T21:09:35.597ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
186992026-03-24T21:09:35.601ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
187002026-03-24T21:09:35.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
187012026-03-24T21:09:35.602ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
187022026-03-24T21:09:35.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
187032026-03-24T21:09:35.604ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187042026-03-24T21:09:35.607ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
187052026-03-24T21:09:35.618ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
187062026-03-24T21:09:35.618ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
187072026-03-24T21:09:35.618ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 35.60466ms iteration = 51 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
187082026-03-24T21:09:35.631ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:338
187092026-03-24T21:09:35.631ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 97.962492ms iteration = 39 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
187102026-03-24T21:09:35.633ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
187112026-03-24T21:09:35.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
187122026-03-24T21:09:35.635ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
187132026-03-24T21:09:35.635ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 77.219472ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
187142026-03-24T21:09:35.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
187152026-03-24T21:09:35.635ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 73.784769ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
187162026-03-24T21:09:35.637ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 75.600449ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
187172026-03-24T21:09:35.637ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 90.713746ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
187182026-03-24T21:09:35.638ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 78.467786ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
187192026-03-24T21:09:35.638ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
187202026-03-24T21:09:35.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
187212026-03-24T21:09:35.643ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:553 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
187222026-03-24T21:09:35.643ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 83.257785ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
187232026-03-24T21:09:35.644ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
187242026-03-24T21:09:35.644ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
187252026-03-24T21:09:35.644ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
187262026-03-24T21:09:35.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
187272026-03-24T21:09:35.645ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
187282026-03-24T21:09:35.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
187292026-03-24T21:09:35.646ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
187302026-03-24T21:09:35.646ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
187312026-03-24T21:09:35.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
187322026-03-24T21:09:35.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
187332026-03-24T21:09:35.648ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187342026-03-24T21:09:35.648ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 86.819971ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
187352026-03-24T21:09:35.650ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
187362026-03-24T21:09:35.650ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
187372026-03-24T21:09:35.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
187382026-03-24T21:09:35.651ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187392026-03-24T21:09:35.654ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
187402026-03-24T21:09:35.655ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
187412026-03-24T21:09:35.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
187422026-03-24T21:09:35.658ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
187432026-03-24T21:09:35.659ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d8f3f20-888f-4260-a2e8-07ae95e0bdbf", "content-length": "691", "date": "Tue, 24 Mar 2026 21:09:35 GMT"} })
187442026-03-24T21:09:35.659ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:119 interfaces = {} sled = buskin
187452026-03-24T21:09:35.659ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:131 mappings = [] sled = buskin
187462026-03-24T21:09:35.659ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:144 mappings = [] sled = buskin
187472026-03-24T21:09:35.659ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 125.845934ms iteration = 26 status = {}
187482026-03-24T21:09:35.690ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 128.327341ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
187492026-03-24T21:09:35.691ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
187502026-03-24T21:09:35.691ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
187512026-03-24T21:09:35.692ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
187522026-03-24T21:09:35.692ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
187532026-03-24T21:09:35.692ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 129.920673ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
187542026-03-24T21:09:35.692ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0752291a-b89a-43b6-81a9-8e5342d90c8e", "content-length": "372", "date": "Tue, 24 Mar 2026 21:09:35 GMT"} })
187552026-03-24T21:09:35.692ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
187562026-03-24T21:09:35.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
187572026-03-24T21:09:35.710ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
187582026-03-24T21:09:35.710ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
187592026-03-24T21:09:35.713ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9418d4b7-7a37-4bf8-8459-c1c3db84eb1d", "date": "Tue, 24 Mar 2026 21:09:35 GMT"} })
187602026-03-24T21:09:35.713ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
187612026-03-24T21:09:35.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
187622026-03-24T21:09:35.716ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
187632026-03-24T21:09:35.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
187642026-03-24T21:09:35.718ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
187652026-03-24T21:09:35.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
187662026-03-24T21:09:35.722ZINFOnexus (BackgroundTasks): VPC route manager sled b4293d4f-3308-4dfa-b261-a103215913e4 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
187672026-03-24T21:09:35.722ZDEBGnexus (BackgroundTasks): client request SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
187682026-03-24T21:09:35.730ZDEBGnexus (BackgroundTasks): client response SledAgent = b4293d4f-3308-4dfa-b261-a103215913e4 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "47586e70-ca40-416a-937b-4df8c2a82420", "date": "Tue, 24 Mar 2026 21:09:35 GMT"} })
187692026-03-24T21:09:35.730ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 168.38855ms iteration = 39 status = {}
187702026-03-24T21:09:35.852ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = ee0b7d07-281d-4353-9052-3357eca0506d result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "9814e4ba-aa6d-45c0-80a9-1a77fbc10f5d", "date": "Tue, 24 Mar 2026 21:09:35 GMT"} })
187712026-03-24T21:09:35.852ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:908 rack_id = ee0b7d07-281d-4353-9052-3357eca0506d switch = switch0
187722026-03-24T21:09:35.852ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
187732026-03-24T21:09:35.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
187742026-03-24T21:09:35.855ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
187752026-03-24T21:09:35.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
187762026-03-24T21:09:35.859ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
187772026-03-24T21:09:35.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
187782026-03-24T21:09:35.861ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
187792026-03-24T21:09:35.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
187802026-03-24T21:09:35.863ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 329.69938ms iteration = 26 status = {}
187812026-03-24T21:09:38.559ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 421 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 9178cf99-e1f9-4d02-90b0-a3a2eeff86d1 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
187822026-03-24T21:09:40.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 152 reason = Timeout
187832026-03-24T21:09:40.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
187842026-03-24T21:09:40.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
187852026-03-24T21:09:40.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187862026-03-24T21:09:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
187872026-03-24T21:09:40.538ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
187882026-03-24T21:09:40.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
187892026-03-24T21:09:40.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
187902026-03-24T21:09:40.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.294697ms iteration = 152 status = {"config_updated":false}
187912026-03-24T21:09:45.533ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
187922026-03-24T21:09:45.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 153 reason = Timeout
187932026-03-24T21:09:45.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
187942026-03-24T21:09:45.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
187952026-03-24T21:09:45.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
187962026-03-24T21:09:45.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
187972026-03-24T21:09:45.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187982026-03-24T21:09:45.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187992026-03-24T21:09:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
188002026-03-24T21:09:45.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
188012026-03-24T21:09:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
188022026-03-24T21:09:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
188032026-03-24T21:09:45.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
188042026-03-24T21:09:45.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
188052026-03-24T21:09:45.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
188062026-03-24T21:09:45.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.418609ms iteration = 153 status = {"config_updated":false}
188072026-03-24T21:09:45.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
188082026-03-24T21:09:45.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
188092026-03-24T21:09:45.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188102026-03-24T21:09:45.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
188112026-03-24T21:09:45.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
188122026-03-24T21:09:45.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
188132026-03-24T21:09:45.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
188142026-03-24T21:09:45.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
188152026-03-24T21:09:45.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
188162026-03-24T21:09:45.563ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
188172026-03-24T21:09:45.563ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.259569ms iteration = 77 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
188182026-03-24T21:09:45.600ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
188192026-03-24T21:09:45.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
188202026-03-24T21:09:45.602ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.736108ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
188212026-03-24T21:09:48.559ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 437 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = af1672e3-dc00-436e-8087-64b61fb15798 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb
188222026-03-24T21:09:50.533ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 68 reason = Timeout
188232026-03-24T21:09:50.533ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 154 reason = Timeout
188242026-03-24T21:09:50.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
188252026-03-24T21:09:50.533ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
188262026-03-24T21:09:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
188272026-03-24T21:09:50.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
188282026-03-24T21:09:50.535ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188292026-03-24T21:09:50.535ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188302026-03-24T21:09:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
188312026-03-24T21:09:50.539ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
188322026-03-24T21:09:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
188332026-03-24T21:09:50.539ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
188342026-03-24T21:09:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
188352026-03-24T21:09:50.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
188362026-03-24T21:09:50.540ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
188372026-03-24T21:09:50.540ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.209364ms iteration = 154 status = {"config_updated":false}
188382026-03-24T21:09:50.541ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 55464b9a-4fc2-45bb-97a5-05ea8d50a065 original_time_started = 2026-03-24 21:09:06.749597 UTC
188392026-03-24T21:09:50.541ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.256593ms iteration = 68 status = {"Loaded":{"collection_id":"55464b9a-4fc2-45bb-97a5-05ea8d50a065","time_loaded":"2026-03-24T21:09:50.533318970Z","time_started":"2026-03-24T21:09:06.749597Z"}}
188402026-03-24T21:09:50.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 52 reason = Timeout
188412026-03-24T21:09:50.582ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 52 reason = Timeout
188422026-03-24T21:09:50.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
188432026-03-24T21:09:50.582ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
188442026-03-24T21:09:50.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
188452026-03-24T21:09:50.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
188462026-03-24T21:09:50.583ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
188472026-03-24T21:09:50.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
188482026-03-24T21:09:50.584ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188492026-03-24T21:09:50.585ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188502026-03-24T21:09:50.586ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
188512026-03-24T21:09:50.587ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
188522026-03-24T21:09:50.587ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
188532026-03-24T21:09:50.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
188542026-03-24T21:09:50.588ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.025282ms iteration = 52 status = "NoSitrep"
188552026-03-24T21:09:50.589ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
188562026-03-24T21:09:50.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
188572026-03-24T21:09:50.589ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
188582026-03-24T21:09:50.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
188592026-03-24T21:09:50.591ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188602026-03-24T21:09:50.592ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
188612026-03-24T21:09:50.595ZINFOnexus (BackgroundTasks): deleted 0 of 0 sitreps alert_requests_deleted = 0 background_task = fm_sitrep_gc case_ereports_deleted = 0 cases_deleted = 0 file = nexus/db-queries/src/db/datastore/fm.rs:988 ids = [] sitreps_deleted = 0
188622026-03-24T21:09:50.595ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
188632026-03-24T21:09:50.595ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.99147ms iteration = 52 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
188642026-03-24T21:09:55.534ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
188652026-03-24T21:09:55.534ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 155 reason = Timeout
188662026-03-24T21:09:55.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
188672026-03-24T21:09:55.534ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
188682026-03-24T21:09:55.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
188692026-03-24T21:09:55.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
188702026-03-24T21:09:55.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188712026-03-24T21:09:55.536ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188722026-03-24T21:09:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
188732026-03-24T21:09:55.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
188742026-03-24T21:09:55.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
188752026-03-24T21:09:55.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
188762026-03-24T21:09:55.540ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
188772026-03-24T21:09:55.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
188782026-03-24T21:09:55.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
188792026-03-24T21:09:55.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
188802026-03-24T21:09:55.541ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
188812026-03-24T21:09:55.541ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.644476ms iteration = 155 status = {"config_updated":false}
188822026-03-24T21:09:55.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188832026-03-24T21:09:55.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9e1c09c1-3960-4ecf-a904-19e137699ab3, lookup_type: ById(9e1c09c1-3960-4ecf-a904-19e137699ab3) } result = Ok(())
188842026-03-24T21:09:55.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
188852026-03-24T21:09:55.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
188862026-03-24T21:09:55.558ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 original_time_created = 2026-03-24 20:56:53.015235 UTC target_id = 9e1c09c1-3960-4ecf-a904-19e137699ab3 time_created = 2026-03-24 20:56:53.015235 UTC
188872026-03-24T21:09:55.558ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.585873ms iteration = 78 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9e1c09c1-3960-4ecf-a904-19e137699ab3","time_created":"2026-03-24T20:56:53.015235Z"}
188882026-03-24T21:09:55.561ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
188892026-03-24T21:09:55.561ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
188902026-03-24T21:09:55.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
188912026-03-24T21:09:55.563ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
188922026-03-24T21:09:55.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
188932026-03-24T21:09:55.565ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.382601ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
188942026-03-24T21:09:58.559ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:867 latency_us = 591 local_addr = [fd00:1122:3344:101::b]:37929 method = GET remote_addr = [fd00:1122:3344:101::d]:59054 req_id = 416d73e6-c399-432b-8cba-c5fa73484762 response_code = 200 uri = /da73ac6f-331d-476f-a388-d3f6ef4cc9cb